Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [................XXXXXX......................................] duration: 500.154855ms FAIL (2025-08-17 13:35:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 500.177437ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................XX.XX....................................] duration: 841.167917ms FAIL (2025-08-17 13:35:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 11h39m18.963301239s) [upload=220201]: Got next segment. Inline: false Aug 17 13:05:38.232064688 (211h39m18.963332307s) [upload=220201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:38.232068765 (211h39m18.963336385s) [upload=220201]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:38.260394616 (211h39m18.991662237s) [seg_pos={0 0}, upload=220201]: Begin upload segment... Aug 17 13:05:38.260422680 (211h39m18.99169031s) [seg_pos={0 0}, upload=220201]: Done begin upload segment. Aug 17 13:05:38.260441515 (211h39m18.991709135s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260457946 (211h39m18.991725576s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260717223 (211h39m18.991984853s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260731841 (211h39m18.991999471s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260959870 (211h39m18.9922275s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260968356 (211h39m18.992235986s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261198057 (211h39m18.992465677s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261233694 (211h39m18.992501334s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261415967 (211h39m18.992683597s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261464098 (211h39m18.992731728s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261631563 (211h39m18.992899193s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261670385 (211h39m18.992938005s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261838482 (211h39m18.993106102s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261906510 (211h39m18.99317414s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262041383 (211h39m18.993309003s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262119430 (211h39m18.993387071s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262267067 (211h39m18.993534687s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262472443 (211h39m18.993740064s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.335674416 (211h39m19.066942037s) [upload=220201]: Next returned nil segment Aug 17 13:05:38.335685137 (211h39m19.066952757s) [upload=220201]: Waiting for error group managing segments... Aug 17 13:05:38.343418127 (211h39m19.074685747s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.343638923 (211h39m19.074906542s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.375313802 (211h39m19.106581432s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.384478504 (211h39m19.115746134s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.394391995 (211h39m19.125659625s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.395010046 (211h39m19.126277676s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.402387698 (211h39m19.133655328s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.409234503 (211h39m19.140502123s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.428228938 (211h39m19.159496568s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.546929112 (211h39m19.278196732s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.562922516 (211h39m19.294190176s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.786308181 (211h39m19.517575811s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.858771665 (211h39m19.590039295s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:06:38.221240067 (211h40m18.952507717s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221311331 (211h40m18.952578981s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:104.36.82.58:4285): protocol: write tcp 5.161.89.252:35328->104.36.82.58:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221354321 (211h40m18.952621971s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12RFJ2jqXCSsXgc8ZDNTptWcaUV5Lm25Eobt58t7vGiQY34uhXz, address:104.36.82.2:4759): protocol: rpc: dial tcp 104.36.82.2:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221450012 (211h40m18.952717662s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf, address:104.36.82.60:4343): protocol: rpc: dial tcp 104.36.82.60:4343: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221534150 (211h40m18.952801799s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12QfYPc1vbgwcnEWk67Bfbt5KiQ8sq322VKe951PFMTF6VaTzAR, address:104.36.82.18:4376): protocol: write tcp 5.161.89.252:49358->104.36.82.18:4376: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221585135 (211h40m18.952852785s) [seg_pos={0 0}, upload=220201]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 13:06:38.236359888 (211h40m18.967627539s) [upload=220201]: Done uploading segments... Aug 17 13:06:38.236387120 (211h40m18.967654769s) [upload=220201]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 841.192552ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.................XXXXXX.....................................] duration: 610.661335ms FAIL (2025-08-17 13:34:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 610.687013ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.........X...XXXX...........................................] duration: 405.553566ms FAIL (2025-08-17 13:34:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 405.572452ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 161.247397ms FAIL (2025-08-17 13:34:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 161.273105ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..................XXXXXX....................................] duration: 358.238168ms FAIL (2025-08-17 13:34:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 358.262533ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..........X...XXXX..........................................] duration: 764.855527ms FAIL (2025-08-17 13:34:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 764.898307ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 178.143335ms FAIL (2025-08-17 13:34:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 178.165627ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...................XXXXXX...................................] duration: 380.446921ms FAIL (2025-08-17 13:34:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 380.487617ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................XX.XX...................................] duration: 604.162472ms FAIL (2025-08-17 13:34:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 11h39m18.963301239s) [upload=220201]: Got next segment. Inline: false Aug 17 13:05:38.232064688 (211h39m18.963332307s) [upload=220201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:38.232068765 (211h39m18.963336385s) [upload=220201]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:38.260394616 (211h39m18.991662237s) [seg_pos={0 0}, upload=220201]: Begin upload segment... Aug 17 13:05:38.260422680 (211h39m18.99169031s) [seg_pos={0 0}, upload=220201]: Done begin upload segment. Aug 17 13:05:38.260441515 (211h39m18.991709135s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260457946 (211h39m18.991725576s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260717223 (211h39m18.991984853s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260731841 (211h39m18.991999471s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260959870 (211h39m18.9922275s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260968356 (211h39m18.992235986s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261198057 (211h39m18.992465677s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261233694 (211h39m18.992501334s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261415967 (211h39m18.992683597s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261464098 (211h39m18.992731728s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261631563 (211h39m18.992899193s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261670385 (211h39m18.992938005s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261838482 (211h39m18.993106102s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261906510 (211h39m18.99317414s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262041383 (211h39m18.993309003s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262119430 (211h39m18.993387071s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262267067 (211h39m18.993534687s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262472443 (211h39m18.993740064s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.335674416 (211h39m19.066942037s) [upload=220201]: Next returned nil segment Aug 17 13:05:38.335685137 (211h39m19.066952757s) [upload=220201]: Waiting for error group managing segments... Aug 17 13:05:38.343418127 (211h39m19.074685747s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.343638923 (211h39m19.074906542s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.375313802 (211h39m19.106581432s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.384478504 (211h39m19.115746134s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.394391995 (211h39m19.125659625s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.395010046 (211h39m19.126277676s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.402387698 (211h39m19.133655328s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.409234503 (211h39m19.140502123s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.428228938 (211h39m19.159496568s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.546929112 (211h39m19.278196732s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.562922516 (211h39m19.294190176s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.786308181 (211h39m19.517575811s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.858771665 (211h39m19.590039295s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:06:38.221240067 (211h40m18.952507717s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221311331 (211h40m18.952578981s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:104.36.82.58:4285): protocol: write tcp 5.161.89.252:35328->104.36.82.58:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221354321 (211h40m18.952621971s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12RFJ2jqXCSsXgc8ZDNTptWcaUV5Lm25Eobt58t7vGiQY34uhXz, address:104.36.82.2:4759): protocol: rpc: dial tcp 104.36.82.2:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221450012 (211h40m18.952717662s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf, address:104.36.82.60:4343): protocol: rpc: dial tcp 104.36.82.60:4343: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221534150 (211h40m18.952801799s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12QfYPc1vbgwcnEWk67Bfbt5KiQ8sq322VKe951PFMTF6VaTzAR, address:104.36.82.18:4376): protocol: write tcp 5.161.89.252:49358->104.36.82.18:4376: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221585135 (211h40m18.952852785s) [seg_pos={0 0}, upload=220201]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 13:06:38.236359888 (211h40m18.967627539s) [upload=220201]: Done uploading segments... Aug 17 13:06:38.236387120 (211h40m18.967654769s) [upload=220201]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 604.187448ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 239.514044ms FAIL (2025-08-17 13:33:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 239.536968ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................XX.XX..................................] duration: 615.171724ms FAIL (2025-08-17 13:33:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 11h39m18.963301239s) [upload=220201]: Got next segment. Inline: false Aug 17 13:05:38.232064688 (211h39m18.963332307s) [upload=220201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:38.232068765 (211h39m18.963336385s) [upload=220201]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:38.260394616 (211h39m18.991662237s) [seg_pos={0 0}, upload=220201]: Begin upload segment... Aug 17 13:05:38.260422680 (211h39m18.99169031s) [seg_pos={0 0}, upload=220201]: Done begin upload segment. Aug 17 13:05:38.260441515 (211h39m18.991709135s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260457946 (211h39m18.991725576s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260717223 (211h39m18.991984853s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260731841 (211h39m18.991999471s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260959870 (211h39m18.9922275s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260968356 (211h39m18.992235986s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261198057 (211h39m18.992465677s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261233694 (211h39m18.992501334s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261415967 (211h39m18.992683597s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261464098 (211h39m18.992731728s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261631563 (211h39m18.992899193s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261670385 (211h39m18.992938005s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261838482 (211h39m18.993106102s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261906510 (211h39m18.99317414s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262041383 (211h39m18.993309003s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262119430 (211h39m18.993387071s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262267067 (211h39m18.993534687s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262472443 (211h39m18.993740064s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.335674416 (211h39m19.066942037s) [upload=220201]: Next returned nil segment Aug 17 13:05:38.335685137 (211h39m19.066952757s) [upload=220201]: Waiting for error group managing segments... Aug 17 13:05:38.343418127 (211h39m19.074685747s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.343638923 (211h39m19.074906542s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.375313802 (211h39m19.106581432s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.384478504 (211h39m19.115746134s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.394391995 (211h39m19.125659625s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.395010046 (211h39m19.126277676s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.402387698 (211h39m19.133655328s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.409234503 (211h39m19.140502123s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.428228938 (211h39m19.159496568s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.546929112 (211h39m19.278196732s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.562922516 (211h39m19.294190176s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.786308181 (211h39m19.517575811s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.858771665 (211h39m19.590039295s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:06:38.221240067 (211h40m18.952507717s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221311331 (211h40m18.952578981s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:104.36.82.58:4285): protocol: write tcp 5.161.89.252:35328->104.36.82.58:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221354321 (211h40m18.952621971s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12RFJ2jqXCSsXgc8ZDNTptWcaUV5Lm25Eobt58t7vGiQY34uhXz, address:104.36.82.2:4759): protocol: rpc: dial tcp 104.36.82.2:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221450012 (211h40m18.952717662s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf, address:104.36.82.60:4343): protocol: rpc: dial tcp 104.36.82.60:4343: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221534150 (211h40m18.952801799s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12QfYPc1vbgwcnEWk67Bfbt5KiQ8sq322VKe951PFMTF6VaTzAR, address:104.36.82.18:4376): protocol: write tcp 5.161.89.252:49358->104.36.82.18:4376: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221585135 (211h40m18.952852785s) [seg_pos={0 0}, upload=220201]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 13:06:38.236359888 (211h40m18.967627539s) [upload=220201]: Done uploading segments... Aug 17 13:06:38.236387120 (211h40m18.967654769s) [upload=220201]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 615.187283ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...........X...XXXX.........................................] duration: 440.031892ms FAIL (2025-08-17 13:33:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 440.071917ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [............X...XXXX........................................] duration: 794.750358ms FAIL (2025-08-17 13:32:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 794.780053ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [....................XXXXXX..................................] duration: 440.288454ms FAIL (2025-08-17 13:32:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 440.310345ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 178.564996ms FAIL (2025-08-17 13:32:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 178.578111ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................XX.XX.................................] duration: 716.344366ms FAIL (2025-08-17 13:32:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 11h39m18.963301239s) [upload=220201]: Got next segment. Inline: false Aug 17 13:05:38.232064688 (211h39m18.963332307s) [upload=220201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:38.232068765 (211h39m18.963336385s) [upload=220201]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:38.260394616 (211h39m18.991662237s) [seg_pos={0 0}, upload=220201]: Begin upload segment... Aug 17 13:05:38.260422680 (211h39m18.99169031s) [seg_pos={0 0}, upload=220201]: Done begin upload segment. Aug 17 13:05:38.260441515 (211h39m18.991709135s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260457946 (211h39m18.991725576s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260717223 (211h39m18.991984853s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260731841 (211h39m18.991999471s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260959870 (211h39m18.9922275s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260968356 (211h39m18.992235986s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261198057 (211h39m18.992465677s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261233694 (211h39m18.992501334s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261415967 (211h39m18.992683597s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261464098 (211h39m18.992731728s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261631563 (211h39m18.992899193s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261670385 (211h39m18.992938005s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261838482 (211h39m18.993106102s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261906510 (211h39m18.99317414s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262041383 (211h39m18.993309003s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262119430 (211h39m18.993387071s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262267067 (211h39m18.993534687s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262472443 (211h39m18.993740064s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.335674416 (211h39m19.066942037s) [upload=220201]: Next returned nil segment Aug 17 13:05:38.335685137 (211h39m19.066952757s) [upload=220201]: Waiting for error group managing segments... Aug 17 13:05:38.343418127 (211h39m19.074685747s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.343638923 (211h39m19.074906542s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.375313802 (211h39m19.106581432s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.384478504 (211h39m19.115746134s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.394391995 (211h39m19.125659625s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.395010046 (211h39m19.126277676s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.402387698 (211h39m19.133655328s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.409234503 (211h39m19.140502123s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.428228938 (211h39m19.159496568s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.546929112 (211h39m19.278196732s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.562922516 (211h39m19.294190176s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.786308181 (211h39m19.517575811s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.858771665 (211h39m19.590039295s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:06:38.221240067 (211h40m18.952507717s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221311331 (211h40m18.952578981s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:104.36.82.58:4285): protocol: write tcp 5.161.89.252:35328->104.36.82.58:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221354321 (211h40m18.952621971s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12RFJ2jqXCSsXgc8ZDNTptWcaUV5Lm25Eobt58t7vGiQY34uhXz, address:104.36.82.2:4759): protocol: rpc: dial tcp 104.36.82.2:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221450012 (211h40m18.952717662s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf, address:104.36.82.60:4343): protocol: rpc: dial tcp 104.36.82.60:4343: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221534150 (211h40m18.952801799s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12QfYPc1vbgwcnEWk67Bfbt5KiQ8sq322VKe951PFMTF6VaTzAR, address:104.36.82.18:4376): protocol: write tcp 5.161.89.252:49358->104.36.82.18:4376: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221585135 (211h40m18.952852785s) [seg_pos={0 0}, upload=220201]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 13:06:38.236359888 (211h40m18.967627539s) [upload=220201]: Done uploading segments... Aug 17 13:06:38.236387120 (211h40m18.967654769s) [upload=220201]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 716.373963ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 161.724413ms FAIL (2025-08-17 13:32:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 161.743279ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.............X...XXXX.......................................] duration: 651.097414ms FAIL (2025-08-17 13:32:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 651.125286ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [X..............X............................................] duration: 339.446705ms FAIL (2025-08-17 13:32:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 339.526364ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....................XXXXXX.................................] duration: 352.749903ms FAIL (2025-08-17 13:32:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 352.774199ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.X..............X...........................................] duration: 238.030293ms FAIL (2025-08-17 13:32:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 238.055321ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................XX.XX................................] duration: 473.461506ms FAIL (2025-08-17 13:31:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 11h39m18.963301239s) [upload=220201]: Got next segment. Inline: false Aug 17 13:05:38.232064688 (211h39m18.963332307s) [upload=220201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:38.232068765 (211h39m18.963336385s) [upload=220201]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:38.260394616 (211h39m18.991662237s) [seg_pos={0 0}, upload=220201]: Begin upload segment... Aug 17 13:05:38.260422680 (211h39m18.99169031s) [seg_pos={0 0}, upload=220201]: Done begin upload segment. Aug 17 13:05:38.260441515 (211h39m18.991709135s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260457946 (211h39m18.991725576s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260717223 (211h39m18.991984853s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260731841 (211h39m18.991999471s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260959870 (211h39m18.9922275s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260968356 (211h39m18.992235986s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261198057 (211h39m18.992465677s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261233694 (211h39m18.992501334s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261415967 (211h39m18.992683597s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261464098 (211h39m18.992731728s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261631563 (211h39m18.992899193s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261670385 (211h39m18.992938005s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261838482 (211h39m18.993106102s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261906510 (211h39m18.99317414s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262041383 (211h39m18.993309003s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262119430 (211h39m18.993387071s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262267067 (211h39m18.993534687s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262472443 (211h39m18.993740064s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.335674416 (211h39m19.066942037s) [upload=220201]: Next returned nil segment Aug 17 13:05:38.335685137 (211h39m19.066952757s) [upload=220201]: Waiting for error group managing segments... Aug 17 13:05:38.343418127 (211h39m19.074685747s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.343638923 (211h39m19.074906542s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.375313802 (211h39m19.106581432s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.384478504 (211h39m19.115746134s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.394391995 (211h39m19.125659625s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.395010046 (211h39m19.126277676s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.402387698 (211h39m19.133655328s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.409234503 (211h39m19.140502123s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.428228938 (211h39m19.159496568s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.546929112 (211h39m19.278196732s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.562922516 (211h39m19.294190176s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.786308181 (211h39m19.517575811s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.858771665 (211h39m19.590039295s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:06:38.221240067 (211h40m18.952507717s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221311331 (211h40m18.952578981s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:104.36.82.58:4285): protocol: write tcp 5.161.89.252:35328->104.36.82.58:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221354321 (211h40m18.952621971s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12RFJ2jqXCSsXgc8ZDNTptWcaUV5Lm25Eobt58t7vGiQY34uhXz, address:104.36.82.2:4759): protocol: rpc: dial tcp 104.36.82.2:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221450012 (211h40m18.952717662s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf, address:104.36.82.60:4343): protocol: rpc: dial tcp 104.36.82.60:4343: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221534150 (211h40m18.952801799s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12QfYPc1vbgwcnEWk67Bfbt5KiQ8sq322VKe951PFMTF6VaTzAR, address:104.36.82.18:4376): protocol: write tcp 5.161.89.252:49358->104.36.82.18:4376: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221585135 (211h40m18.952852785s) [seg_pos={0 0}, upload=220201]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 13:06:38.236359888 (211h40m18.967627539s) [upload=220201]: Done uploading segments... Aug 17 13:06:38.236387120 (211h40m18.967654769s) [upload=220201]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 473.49583ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..............X...XXXX......................................] duration: 472.559967ms FAIL (2025-08-17 13:31:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 472.585596ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [......................XXXXXX................................] duration: 402.64874ms FAIL (2025-08-17 13:31:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 402.673817ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..X..............X..........................................] duration: 144.75059ms FAIL (2025-08-17 13:31:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 144.764937ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................XX.XX...............................] duration: 448.694238ms FAIL (2025-08-17 13:30:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 11h39m18.963301239s) [upload=220201]: Got next segment. Inline: false Aug 17 13:05:38.232064688 (211h39m18.963332307s) [upload=220201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:38.232068765 (211h39m18.963336385s) [upload=220201]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:38.260394616 (211h39m18.991662237s) [seg_pos={0 0}, upload=220201]: Begin upload segment... Aug 17 13:05:38.260422680 (211h39m18.99169031s) [seg_pos={0 0}, upload=220201]: Done begin upload segment. Aug 17 13:05:38.260441515 (211h39m18.991709135s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260457946 (211h39m18.991725576s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260717223 (211h39m18.991984853s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260731841 (211h39m18.991999471s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260959870 (211h39m18.9922275s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260968356 (211h39m18.992235986s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261198057 (211h39m18.992465677s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261233694 (211h39m18.992501334s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261415967 (211h39m18.992683597s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261464098 (211h39m18.992731728s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261631563 (211h39m18.992899193s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261670385 (211h39m18.992938005s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261838482 (211h39m18.993106102s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261906510 (211h39m18.99317414s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262041383 (211h39m18.993309003s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262119430 (211h39m18.993387071s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262267067 (211h39m18.993534687s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262472443 (211h39m18.993740064s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.335674416 (211h39m19.066942037s) [upload=220201]: Next returned nil segment Aug 17 13:05:38.335685137 (211h39m19.066952757s) [upload=220201]: Waiting for error group managing segments... Aug 17 13:05:38.343418127 (211h39m19.074685747s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.343638923 (211h39m19.074906542s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.375313802 (211h39m19.106581432s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.384478504 (211h39m19.115746134s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.394391995 (211h39m19.125659625s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.395010046 (211h39m19.126277676s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.402387698 (211h39m19.133655328s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.409234503 (211h39m19.140502123s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.428228938 (211h39m19.159496568s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.546929112 (211h39m19.278196732s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.562922516 (211h39m19.294190176s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.786308181 (211h39m19.517575811s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.858771665 (211h39m19.590039295s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:06:38.221240067 (211h40m18.952507717s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221311331 (211h40m18.952578981s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:104.36.82.58:4285): protocol: write tcp 5.161.89.252:35328->104.36.82.58:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221354321 (211h40m18.952621971s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12RFJ2jqXCSsXgc8ZDNTptWcaUV5Lm25Eobt58t7vGiQY34uhXz, address:104.36.82.2:4759): protocol: rpc: dial tcp 104.36.82.2:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221450012 (211h40m18.952717662s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf, address:104.36.82.60:4343): protocol: rpc: dial tcp 104.36.82.60:4343: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221534150 (211h40m18.952801799s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12QfYPc1vbgwcnEWk67Bfbt5KiQ8sq322VKe951PFMTF6VaTzAR, address:104.36.82.18:4376): protocol: write tcp 5.161.89.252:49358->104.36.82.18:4376: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221585135 (211h40m18.952852785s) [seg_pos={0 0}, upload=220201]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 13:06:38.236359888 (211h40m18.967627539s) [upload=220201]: Done uploading segments... Aug 17 13:06:38.236387120 (211h40m18.967654769s) [upload=220201]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 448.723443ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.......................XXXXXX...............................] duration: 620.220219ms FAIL (2025-08-17 13:30:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 620.274811ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...............X...XXXX.....................................] duration: 548.274505ms FAIL (2025-08-17 13:30:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 548.294515ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...X..............X.........................................] duration: 266.495114ms FAIL (2025-08-17 13:30:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 266.516676ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........................XX.XX..............................] duration: 489.014235ms FAIL (2025-08-17 13:29:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 11h39m18.963301239s) [upload=220201]: Got next segment. Inline: false Aug 17 13:05:38.232064688 (211h39m18.963332307s) [upload=220201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:38.232068765 (211h39m18.963336385s) [upload=220201]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:38.260394616 (211h39m18.991662237s) [seg_pos={0 0}, upload=220201]: Begin upload segment... Aug 17 13:05:38.260422680 (211h39m18.99169031s) [seg_pos={0 0}, upload=220201]: Done begin upload segment. Aug 17 13:05:38.260441515 (211h39m18.991709135s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260457946 (211h39m18.991725576s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260717223 (211h39m18.991984853s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260731841 (211h39m18.991999471s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260959870 (211h39m18.9922275s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260968356 (211h39m18.992235986s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261198057 (211h39m18.992465677s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261233694 (211h39m18.992501334s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261415967 (211h39m18.992683597s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261464098 (211h39m18.992731728s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261631563 (211h39m18.992899193s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261670385 (211h39m18.992938005s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261838482 (211h39m18.993106102s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261906510 (211h39m18.99317414s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262041383 (211h39m18.993309003s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262119430 (211h39m18.993387071s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262267067 (211h39m18.993534687s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262472443 (211h39m18.993740064s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.335674416 (211h39m19.066942037s) [upload=220201]: Next returned nil segment Aug 17 13:05:38.335685137 (211h39m19.066952757s) [upload=220201]: Waiting for error group managing segments... Aug 17 13:05:38.343418127 (211h39m19.074685747s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.343638923 (211h39m19.074906542s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.375313802 (211h39m19.106581432s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.384478504 (211h39m19.115746134s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.394391995 (211h39m19.125659625s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.395010046 (211h39m19.126277676s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.402387698 (211h39m19.133655328s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.409234503 (211h39m19.140502123s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.428228938 (211h39m19.159496568s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.546929112 (211h39m19.278196732s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.562922516 (211h39m19.294190176s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.786308181 (211h39m19.517575811s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.858771665 (211h39m19.590039295s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:06:38.221240067 (211h40m18.952507717s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221311331 (211h40m18.952578981s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:104.36.82.58:4285): protocol: write tcp 5.161.89.252:35328->104.36.82.58:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221354321 (211h40m18.952621971s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12RFJ2jqXCSsXgc8ZDNTptWcaUV5Lm25Eobt58t7vGiQY34uhXz, address:104.36.82.2:4759): protocol: rpc: dial tcp 104.36.82.2:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221450012 (211h40m18.952717662s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf, address:104.36.82.60:4343): protocol: rpc: dial tcp 104.36.82.60:4343: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221534150 (211h40m18.952801799s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12QfYPc1vbgwcnEWk67Bfbt5KiQ8sq322VKe951PFMTF6VaTzAR, address:104.36.82.18:4376): protocol: write tcp 5.161.89.252:49358->104.36.82.18:4376: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221585135 (211h40m18.952852785s) [seg_pos={0 0}, upload=220201]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 13:06:38.236359888 (211h40m18.967627539s) [upload=220201]: Done uploading segments... Aug 17 13:06:38.236387120 (211h40m18.967654769s) [upload=220201]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 489.034223ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....X..............X........................................] duration: 241.145442ms FAIL (2025-08-17 13:29:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 241.167092ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [........................XXXXXX..............................] duration: 369.322843ms FAIL (2025-08-17 13:29:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 369.34795ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [................X...XXXX....................................] duration: 650.110927ms FAIL (2025-08-17 13:29:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 650.129973ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........................XX.XX.............................] duration: 731.524877ms FAIL (2025-08-17 13:29:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 11h39m18.963301239s) [upload=220201]: Got next segment. Inline: false Aug 17 13:05:38.232064688 (211h39m18.963332307s) [upload=220201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:38.232068765 (211h39m18.963336385s) [upload=220201]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:38.260394616 (211h39m18.991662237s) [seg_pos={0 0}, upload=220201]: Begin upload segment... Aug 17 13:05:38.260422680 (211h39m18.99169031s) [seg_pos={0 0}, upload=220201]: Done begin upload segment. Aug 17 13:05:38.260441515 (211h39m18.991709135s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260457946 (211h39m18.991725576s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260717223 (211h39m18.991984853s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260731841 (211h39m18.991999471s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260959870 (211h39m18.9922275s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260968356 (211h39m18.992235986s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261198057 (211h39m18.992465677s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261233694 (211h39m18.992501334s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261415967 (211h39m18.992683597s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261464098 (211h39m18.992731728s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261631563 (211h39m18.992899193s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261670385 (211h39m18.992938005s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261838482 (211h39m18.993106102s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261906510 (211h39m18.99317414s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262041383 (211h39m18.993309003s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262119430 (211h39m18.993387071s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262267067 (211h39m18.993534687s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262472443 (211h39m18.993740064s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.335674416 (211h39m19.066942037s) [upload=220201]: Next returned nil segment Aug 17 13:05:38.335685137 (211h39m19.066952757s) [upload=220201]: Waiting for error group managing segments... Aug 17 13:05:38.343418127 (211h39m19.074685747s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.343638923 (211h39m19.074906542s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.375313802 (211h39m19.106581432s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.384478504 (211h39m19.115746134s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.394391995 (211h39m19.125659625s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.395010046 (211h39m19.126277676s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.402387698 (211h39m19.133655328s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.409234503 (211h39m19.140502123s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.428228938 (211h39m19.159496568s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.546929112 (211h39m19.278196732s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.562922516 (211h39m19.294190176s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.786308181 (211h39m19.517575811s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.858771665 (211h39m19.590039295s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:06:38.221240067 (211h40m18.952507717s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221311331 (211h40m18.952578981s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:104.36.82.58:4285): protocol: write tcp 5.161.89.252:35328->104.36.82.58:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221354321 (211h40m18.952621971s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12RFJ2jqXCSsXgc8ZDNTptWcaUV5Lm25Eobt58t7vGiQY34uhXz, address:104.36.82.2:4759): protocol: rpc: dial tcp 104.36.82.2:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221450012 (211h40m18.952717662s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf, address:104.36.82.60:4343): protocol: rpc: dial tcp 104.36.82.60:4343: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221534150 (211h40m18.952801799s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12QfYPc1vbgwcnEWk67Bfbt5KiQ8sq322VKe951PFMTF6VaTzAR, address:104.36.82.18:4376): protocol: write tcp 5.161.89.252:49358->104.36.82.18:4376: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221585135 (211h40m18.952852785s) [seg_pos={0 0}, upload=220201]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 13:06:38.236359888 (211h40m18.967627539s) [upload=220201]: Done uploading segments... Aug 17 13:06:38.236387120 (211h40m18.967654769s) [upload=220201]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 731.546168ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....X..............X.......................................] duration: 145.405105ms FAIL (2025-08-17 13:29:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 145.447727ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.........................XXXXXX.............................] duration: 396.800117ms FAIL (2025-08-17 13:29:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 396.819182ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........................XX.XX............................] duration: 731.184055ms FAIL (2025-08-17 13:28:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 11h39m18.963301239s) [upload=220201]: Got next segment. Inline: false Aug 17 13:05:38.232064688 (211h39m18.963332307s) [upload=220201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:38.232068765 (211h39m18.963336385s) [upload=220201]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:38.260394616 (211h39m18.991662237s) [seg_pos={0 0}, upload=220201]: Begin upload segment... Aug 17 13:05:38.260422680 (211h39m18.99169031s) [seg_pos={0 0}, upload=220201]: Done begin upload segment. Aug 17 13:05:38.260441515 (211h39m18.991709135s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260457946 (211h39m18.991725576s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260717223 (211h39m18.991984853s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260731841 (211h39m18.991999471s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260959870 (211h39m18.9922275s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260968356 (211h39m18.992235986s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261198057 (211h39m18.992465677s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261233694 (211h39m18.992501334s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261415967 (211h39m18.992683597s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261464098 (211h39m18.992731728s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261631563 (211h39m18.992899193s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261670385 (211h39m18.992938005s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261838482 (211h39m18.993106102s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261906510 (211h39m18.99317414s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262041383 (211h39m18.993309003s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262119430 (211h39m18.993387071s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262267067 (211h39m18.993534687s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262472443 (211h39m18.993740064s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.335674416 (211h39m19.066942037s) [upload=220201]: Next returned nil segment Aug 17 13:05:38.335685137 (211h39m19.066952757s) [upload=220201]: Waiting for error group managing segments... Aug 17 13:05:38.343418127 (211h39m19.074685747s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.343638923 (211h39m19.074906542s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.375313802 (211h39m19.106581432s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.384478504 (211h39m19.115746134s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.394391995 (211h39m19.125659625s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.395010046 (211h39m19.126277676s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.402387698 (211h39m19.133655328s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.409234503 (211h39m19.140502123s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.428228938 (211h39m19.159496568s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.546929112 (211h39m19.278196732s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.562922516 (211h39m19.294190176s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.786308181 (211h39m19.517575811s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.858771665 (211h39m19.590039295s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:06:38.221240067 (211h40m18.952507717s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221311331 (211h40m18.952578981s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:104.36.82.58:4285): protocol: write tcp 5.161.89.252:35328->104.36.82.58:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221354321 (211h40m18.952621971s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12RFJ2jqXCSsXgc8ZDNTptWcaUV5Lm25Eobt58t7vGiQY34uhXz, address:104.36.82.2:4759): protocol: rpc: dial tcp 104.36.82.2:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221450012 (211h40m18.952717662s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf, address:104.36.82.60:4343): protocol: rpc: dial tcp 104.36.82.60:4343: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221534150 (211h40m18.952801799s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12QfYPc1vbgwcnEWk67Bfbt5KiQ8sq322VKe951PFMTF6VaTzAR, address:104.36.82.18:4376): protocol: write tcp 5.161.89.252:49358->104.36.82.18:4376: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221585135 (211h40m18.952852785s) [seg_pos={0 0}, upload=220201]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 13:06:38.236359888 (211h40m18.967627539s) [upload=220201]: Done uploading segments... Aug 17 13:06:38.236387120 (211h40m18.967654769s) [upload=220201]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 731.244548ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.................X...XXXX...................................] duration: 1.257219151s FAIL (2025-08-17 13:28:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1.257243478s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [......X..............X......................................] duration: 134.515883ms FAIL (2025-08-17 13:28:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 134.536572ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..........................XXXXXX............................] duration: 369.61543ms FAIL (2025-08-17 13:28:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 369.632482ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......X..............X.....................................] duration: 172.530564ms FAIL (2025-08-17 13:28:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 172.551483ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..................X...XXXX..................................] duration: 559.401932ms FAIL (2025-08-17 13:28:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 559.434513ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [............................XX.XX...........................] duration: 467.089578ms FAIL (2025-08-17 13:28:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 11h39m18.963301239s) [upload=220201]: Got next segment. Inline: false Aug 17 13:05:38.232064688 (211h39m18.963332307s) [upload=220201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:38.232068765 (211h39m18.963336385s) [upload=220201]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:38.260394616 (211h39m18.991662237s) [seg_pos={0 0}, upload=220201]: Begin upload segment... Aug 17 13:05:38.260422680 (211h39m18.99169031s) [seg_pos={0 0}, upload=220201]: Done begin upload segment. Aug 17 13:05:38.260441515 (211h39m18.991709135s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260457946 (211h39m18.991725576s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260717223 (211h39m18.991984853s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260731841 (211h39m18.991999471s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260959870 (211h39m18.9922275s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260968356 (211h39m18.992235986s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261198057 (211h39m18.992465677s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261233694 (211h39m18.992501334s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261415967 (211h39m18.992683597s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261464098 (211h39m18.992731728s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261631563 (211h39m18.992899193s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261670385 (211h39m18.992938005s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261838482 (211h39m18.993106102s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261906510 (211h39m18.99317414s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262041383 (211h39m18.993309003s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262119430 (211h39m18.993387071s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262267067 (211h39m18.993534687s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262472443 (211h39m18.993740064s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.335674416 (211h39m19.066942037s) [upload=220201]: Next returned nil segment Aug 17 13:05:38.335685137 (211h39m19.066952757s) [upload=220201]: Waiting for error group managing segments... Aug 17 13:05:38.343418127 (211h39m19.074685747s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.343638923 (211h39m19.074906542s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.375313802 (211h39m19.106581432s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.384478504 (211h39m19.115746134s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.394391995 (211h39m19.125659625s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.395010046 (211h39m19.126277676s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.402387698 (211h39m19.133655328s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.409234503 (211h39m19.140502123s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.428228938 (211h39m19.159496568s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.546929112 (211h39m19.278196732s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.562922516 (211h39m19.294190176s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.786308181 (211h39m19.517575811s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.858771665 (211h39m19.590039295s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:06:38.221240067 (211h40m18.952507717s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221311331 (211h40m18.952578981s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:104.36.82.58:4285): protocol: write tcp 5.161.89.252:35328->104.36.82.58:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221354321 (211h40m18.952621971s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12RFJ2jqXCSsXgc8ZDNTptWcaUV5Lm25Eobt58t7vGiQY34uhXz, address:104.36.82.2:4759): protocol: rpc: dial tcp 104.36.82.2:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221450012 (211h40m18.952717662s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf, address:104.36.82.60:4343): protocol: rpc: dial tcp 104.36.82.60:4343: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221534150 (211h40m18.952801799s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12QfYPc1vbgwcnEWk67Bfbt5KiQ8sq322VKe951PFMTF6VaTzAR, address:104.36.82.18:4376): protocol: write tcp 5.161.89.252:49358->104.36.82.18:4376: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221585135 (211h40m18.952852785s) [seg_pos={0 0}, upload=220201]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 13:06:38.236359888 (211h40m18.967627539s) [upload=220201]: Done uploading segments... Aug 17 13:06:38.236387120 (211h40m18.967654769s) [upload=220201]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 467.117631ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...........................XXXXXX...........................] duration: 428.361555ms FAIL (2025-08-17 13:27:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 428.379849ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [............................XXXXXX..........................] duration: 417.515219ms FAIL (2025-08-17 13:27:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 417.542892ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........X..............X....................................] duration: 581.292056ms FAIL (2025-08-17 13:27:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 581.315359ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...................X...XXXX.................................] duration: 507.487532ms FAIL (2025-08-17 13:27:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 507.510926ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........X..............X...................................] duration: 172.823113ms FAIL (2025-08-17 13:26:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 172.842099ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............................XX.XX..........................] duration: 434.445202ms FAIL (2025-08-17 13:26:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 11h39m18.963301239s) [upload=220201]: Got next segment. Inline: false Aug 17 13:05:38.232064688 (211h39m18.963332307s) [upload=220201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:38.232068765 (211h39m18.963336385s) [upload=220201]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:38.260394616 (211h39m18.991662237s) [seg_pos={0 0}, upload=220201]: Begin upload segment... Aug 17 13:05:38.260422680 (211h39m18.99169031s) [seg_pos={0 0}, upload=220201]: Done begin upload segment. Aug 17 13:05:38.260441515 (211h39m18.991709135s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260457946 (211h39m18.991725576s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260717223 (211h39m18.991984853s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260731841 (211h39m18.991999471s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260959870 (211h39m18.9922275s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260968356 (211h39m18.992235986s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261198057 (211h39m18.992465677s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261233694 (211h39m18.992501334s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261415967 (211h39m18.992683597s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261464098 (211h39m18.992731728s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261631563 (211h39m18.992899193s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261670385 (211h39m18.992938005s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261838482 (211h39m18.993106102s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261906510 (211h39m18.99317414s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262041383 (211h39m18.993309003s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262119430 (211h39m18.993387071s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262267067 (211h39m18.993534687s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262472443 (211h39m18.993740064s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.335674416 (211h39m19.066942037s) [upload=220201]: Next returned nil segment Aug 17 13:05:38.335685137 (211h39m19.066952757s) [upload=220201]: Waiting for error group managing segments... Aug 17 13:05:38.343418127 (211h39m19.074685747s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.343638923 (211h39m19.074906542s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.375313802 (211h39m19.106581432s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.384478504 (211h39m19.115746134s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.394391995 (211h39m19.125659625s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.395010046 (211h39m19.126277676s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.402387698 (211h39m19.133655328s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.409234503 (211h39m19.140502123s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.428228938 (211h39m19.159496568s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.546929112 (211h39m19.278196732s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.562922516 (211h39m19.294190176s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.786308181 (211h39m19.517575811s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.858771665 (211h39m19.590039295s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:06:38.221240067 (211h40m18.952507717s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221311331 (211h40m18.952578981s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:104.36.82.58:4285): protocol: write tcp 5.161.89.252:35328->104.36.82.58:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221354321 (211h40m18.952621971s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12RFJ2jqXCSsXgc8ZDNTptWcaUV5Lm25Eobt58t7vGiQY34uhXz, address:104.36.82.2:4759): protocol: rpc: dial tcp 104.36.82.2:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221450012 (211h40m18.952717662s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf, address:104.36.82.60:4343): protocol: rpc: dial tcp 104.36.82.60:4343: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221534150 (211h40m18.952801799s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12QfYPc1vbgwcnEWk67Bfbt5KiQ8sq322VKe951PFMTF6VaTzAR, address:104.36.82.18:4376): protocol: write tcp 5.161.89.252:49358->104.36.82.18:4376: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221585135 (211h40m18.952852785s) [seg_pos={0 0}, upload=220201]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 13:06:38.236359888 (211h40m18.967627539s) [upload=220201]: Done uploading segments... Aug 17 13:06:38.236387120 (211h40m18.967654769s) [upload=220201]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 434.473715ms core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)