Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................................................XX.XX..] duration: 565.145341ms FAIL (2025-08-17 13:07:38 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 11h39m18.963301239s) [upload=220201]: Got next segment. Inline: false Aug 17 13:05:38.232064688 (211h39m18.963332307s) [upload=220201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:38.232068765 (211h39m18.963336385s) [upload=220201]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:38.260394616 (211h39m18.991662237s) [seg_pos={0 0}, upload=220201]: Begin upload segment... Aug 17 13:05:38.260422680 (211h39m18.99169031s) [seg_pos={0 0}, upload=220201]: Done begin upload segment. Aug 17 13:05:38.260441515 (211h39m18.991709135s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260457946 (211h39m18.991725576s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260717223 (211h39m18.991984853s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260731841 (211h39m18.991999471s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260959870 (211h39m18.9922275s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260968356 (211h39m18.992235986s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261198057 (211h39m18.992465677s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261233694 (211h39m18.992501334s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261415967 (211h39m18.992683597s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261464098 (211h39m18.992731728s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261631563 (211h39m18.992899193s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261670385 (211h39m18.992938005s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261838482 (211h39m18.993106102s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261906510 (211h39m18.99317414s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262041383 (211h39m18.993309003s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262119430 (211h39m18.993387071s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262267067 (211h39m18.993534687s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262472443 (211h39m18.993740064s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.335674416 (211h39m19.066942037s) [upload=220201]: Next returned nil segment Aug 17 13:05:38.335685137 (211h39m19.066952757s) [upload=220201]: Waiting for error group managing segments... Aug 17 13:05:38.343418127 (211h39m19.074685747s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.343638923 (211h39m19.074906542s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.375313802 (211h39m19.106581432s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.384478504 (211h39m19.115746134s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.394391995 (211h39m19.125659625s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.395010046 (211h39m19.126277676s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.402387698 (211h39m19.133655328s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.409234503 (211h39m19.140502123s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.428228938 (211h39m19.159496568s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.546929112 (211h39m19.278196732s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.562922516 (211h39m19.294190176s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.786308181 (211h39m19.517575811s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.858771665 (211h39m19.590039295s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:06:38.221240067 (211h40m18.952507717s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221311331 (211h40m18.952578981s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:104.36.82.58:4285): protocol: write tcp 5.161.89.252:35328->104.36.82.58:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221354321 (211h40m18.952621971s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12RFJ2jqXCSsXgc8ZDNTptWcaUV5Lm25Eobt58t7vGiQY34uhXz, address:104.36.82.2:4759): protocol: rpc: dial tcp 104.36.82.2:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221450012 (211h40m18.952717662s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf, address:104.36.82.60:4343): protocol: rpc: dial tcp 104.36.82.60:4343: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221534150 (211h40m18.952801799s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12QfYPc1vbgwcnEWk67Bfbt5KiQ8sq322VKe951PFMTF6VaTzAR, address:104.36.82.18:4376): protocol: write tcp 5.161.89.252:49358->104.36.82.18:4376: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221585135 (211h40m18.952852785s) [seg_pos={0 0}, upload=220201]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 13:06:38.236359888 (211h40m18.967627539s) [upload=220201]: Done uploading segments... Aug 17 13:06:38.236387120 (211h40m18.967654769s) [upload=220201]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 565.168223ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...................................X..............X.........] duration: 365.7001ms FAIL (2025-08-17 13:07:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 365.740045ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................................................XX.XX.] duration: 48.31290388s FAIL (2025-08-17 13:07:26 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 11h39m18.963301239s) [upload=220201]: Got next segment. Inline: false Aug 17 13:05:38.232064688 (211h39m18.963332307s) [upload=220201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:38.232068765 (211h39m18.963336385s) [upload=220201]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:38.260394616 (211h39m18.991662237s) [seg_pos={0 0}, upload=220201]: Begin upload segment... Aug 17 13:05:38.260422680 (211h39m18.99169031s) [seg_pos={0 0}, upload=220201]: Done begin upload segment. Aug 17 13:05:38.260441515 (211h39m18.991709135s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260457946 (211h39m18.991725576s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260717223 (211h39m18.991984853s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260731841 (211h39m18.991999471s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260959870 (211h39m18.9922275s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260968356 (211h39m18.992235986s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261198057 (211h39m18.992465677s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261233694 (211h39m18.992501334s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261415967 (211h39m18.992683597s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261464098 (211h39m18.992731728s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261631563 (211h39m18.992899193s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261670385 (211h39m18.992938005s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261838482 (211h39m18.993106102s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261906510 (211h39m18.99317414s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262041383 (211h39m18.993309003s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262119430 (211h39m18.993387071s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262267067 (211h39m18.993534687s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262472443 (211h39m18.993740064s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.335674416 (211h39m19.066942037s) [upload=220201]: Next returned nil segment Aug 17 13:05:38.335685137 (211h39m19.066952757s) [upload=220201]: Waiting for error group managing segments... Aug 17 13:05:38.343418127 (211h39m19.074685747s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.343638923 (211h39m19.074906542s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.375313802 (211h39m19.106581432s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.384478504 (211h39m19.115746134s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.394391995 (211h39m19.125659625s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.395010046 (211h39m19.126277676s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.402387698 (211h39m19.133655328s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.409234503 (211h39m19.140502123s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.428228938 (211h39m19.159496568s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.546929112 (211h39m19.278196732s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.562922516 (211h39m19.294190176s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.786308181 (211h39m19.517575811s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.858771665 (211h39m19.590039295s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:06:38.221240067 (211h40m18.952507717s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221311331 (211h40m18.952578981s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:104.36.82.58:4285): protocol: write tcp 5.161.89.252:35328->104.36.82.58:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221354321 (211h40m18.952621971s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12RFJ2jqXCSsXgc8ZDNTptWcaUV5Lm25Eobt58t7vGiQY34uhXz, address:104.36.82.2:4759): protocol: rpc: dial tcp 104.36.82.2:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221450012 (211h40m18.952717662s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf, address:104.36.82.60:4343): protocol: rpc: dial tcp 104.36.82.60:4343: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221534150 (211h40m18.952801799s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12QfYPc1vbgwcnEWk67Bfbt5KiQ8sq322VKe951PFMTF6VaTzAR, address:104.36.82.18:4376): protocol: write tcp 5.161.89.252:49358->104.36.82.18:4376: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221585135 (211h40m18.952852785s) [seg_pos={0 0}, upload=220201]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 13:06:38.236359888 (211h40m18.967627539s) [upload=220201]: Done uploading segments... Aug 17 13:06:38.236387120 (211h40m18.967654769s) [upload=220201]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 48.312922816s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [....................................................XXXXXX..] duration: 7.610040799s FAIL (2025-08-17 13:07:18 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 7.610067188s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [X................................................X...XXXX...] duration: 5.651349984s FAIL (2025-08-17 13:07:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.651380852s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....................................X..............X........] duration: 771.277336ms FAIL (2025-08-17 13:06:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 771.297484ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................................................XX.XX] duration: 1m0.017373852s FAIL (2025-08-17 13:06:38 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 11h39m18.963301239s) [upload=220201]: Got next segment. Inline: false Aug 17 13:05:38.232064688 (211h39m18.963332307s) [upload=220201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:38.232068765 (211h39m18.963336385s) [upload=220201]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:38.260394616 (211h39m18.991662237s) [seg_pos={0 0}, upload=220201]: Begin upload segment... Aug 17 13:05:38.260422680 (211h39m18.99169031s) [seg_pos={0 0}, upload=220201]: Done begin upload segment. Aug 17 13:05:38.260441515 (211h39m18.991709135s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260457946 (211h39m18.991725576s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260717223 (211h39m18.991984853s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260731841 (211h39m18.991999471s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260959870 (211h39m18.9922275s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260968356 (211h39m18.992235986s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261198057 (211h39m18.992465677s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261233694 (211h39m18.992501334s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261415967 (211h39m18.992683597s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261464098 (211h39m18.992731728s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261631563 (211h39m18.992899193s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261670385 (211h39m18.992938005s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261838482 (211h39m18.993106102s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261906510 (211h39m18.99317414s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262041383 (211h39m18.993309003s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262119430 (211h39m18.993387071s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262267067 (211h39m18.993534687s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262472443 (211h39m18.993740064s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.335674416 (211h39m19.066942037s) [upload=220201]: Next returned nil segment Aug 17 13:05:38.335685137 (211h39m19.066952757s) [upload=220201]: Waiting for error group managing segments... Aug 17 13:05:38.343418127 (211h39m19.074685747s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.343638923 (211h39m19.074906542s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.375313802 (211h39m19.106581432s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.384478504 (211h39m19.115746134s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.394391995 (211h39m19.125659625s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.395010046 (211h39m19.126277676s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.402387698 (211h39m19.133655328s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.409234503 (211h39m19.140502123s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.428228938 (211h39m19.159496568s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.546929112 (211h39m19.278196732s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.562922516 (211h39m19.294190176s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.786308181 (211h39m19.517575811s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.858771665 (211h39m19.590039295s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:06:38.221240067 (211h40m18.952507717s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221311331 (211h40m18.952578981s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:104.36.82.58:4285): protocol: write tcp 5.161.89.252:35328->104.36.82.58:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221354321 (211h40m18.952621971s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12RFJ2jqXCSsXgc8ZDNTptWcaUV5Lm25Eobt58t7vGiQY34uhXz, address:104.36.82.2:4759): protocol: rpc: dial tcp 104.36.82.2:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221450012 (211h40m18.952717662s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf, address:104.36.82.60:4343): protocol: rpc: dial tcp 104.36.82.60:4343: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221534150 (211h40m18.952801799s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12QfYPc1vbgwcnEWk67Bfbt5KiQ8sq322VKe951PFMTF6VaTzAR, address:104.36.82.18:4376): protocol: write tcp 5.161.89.252:49358->104.36.82.18:4376: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221585135 (211h40m18.952852785s) [seg_pos={0 0}, upload=220201]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 13:06:38.236359888 (211h40m18.967627539s) [upload=220201]: Done uploading segments... Aug 17 13:06:38.236387120 (211h40m18.967654769s) [upload=220201]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.017405922s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.X................................................X...XXXX..] duration: 977.5318ms FAIL (2025-08-17 13:06:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 977.556556ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X..............X.......] duration: 156.993781ms FAIL (2025-08-17 13:06:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 157.015181ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....................................................XXXXXX.] duration: 438.047979ms FAIL (2025-08-17 13:06:12 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 438.067857ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [..X................................................X...XXXX.] duration: 717.713689ms FAIL (2025-08-17 13:06:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 717.731283ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [...X................................................X...XXXX] duration: 1m0.028136791s FAIL (2025-08-17 13:06:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1m0.028181215s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration: 30.018972032s FAIL (2025-08-17 13:05:56 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.018985507s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [......................................X..............X......] duration: 388.189137ms FAIL (2025-08-17 13:05:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 388.216858ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................................................XX.X] duration: 1m0.015812818s FAIL (2025-08-17 13:05:38 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== ]: Uploading piece... Aug 17 13:04:38.193247793 (211h38m18.924515404s) [noise=true , address=104.36.82.5:4849 , piece_id=CSB4SA3ENRUUCPNXHTDV6IQPURPAR4OC27XW4MZ4LX6RHSRQGA5Q, seg_pos={0 0}, upload=220186]: Uploading piece... Aug 17 13:04:38.193254075 (211h38m18.924521696s) [noise=true , address=79.127.220.101:4041 , piece_id=AIMGHT5ZZ6SU6MJDZ55T24K67MTU6LBPQC47QXODOEDRLSAXTSHA, seg_pos={0 0}, upload=220186]: Uploading piece... Aug 17 13:04:38.193507712 (211h38m18.924775332s) [noise=true , address=45.91.116.10:4110 , piece_id=GIAG53YHED3CHN3GM37K2VMHSDO4CWKIHGX2R6E7WKTCX3SD6ICQ, seg_pos={0 0}, upload=220186]: Uploading piece... Aug 17 13:04:38.193547467 (211h38m18.924815097s) [noise=true , address=104.36.82.59:4317 , piece_id=EOJBVJGNO4MJ6IRE7DGBH43UBWYOAV3L5LTKUXTFJGSHMZDWPLMA, seg_pos={0 0}, upload=220186]: Uploading piece... Aug 17 13:04:38.193750669 (211h38m18.925018289s) [noise=true , address=104.36.82.36:4626 , piece_id=53TRPZJPSOBJOYDCTJTEBO7IPYED2R57KDTXYW2KEFLCVT32HB7Q, seg_pos={0 0}, upload=220186]: Uploading piece... Aug 17 13:04:38.193820149 (211h38m18.925087779s) [noise=true , address=104.36.82.12:4169 , piece_id=E6Q46W63HR4LX4OVTT4RW5GGML42Q5EKX7PNNFPKIWMDWOUJITOQ, seg_pos={0 0}, upload=220186]: Uploading piece... Aug 17 13:04:38.193987274 (211h38m18.925254893s) [noise=true , address=104.36.82.60:4353 , piece_id=337YPO4U6YEG5SYKOU3E5LGZMH56FQKW4S7WIZBLBPIEA3DH42KA, seg_pos={0 0}, upload=220186]: Uploading piece... Aug 17 13:04:38.194158886 (211h38m18.925426517s) [noise=true , address=104.36.82.17:4331 , piece_id=7QWYB6C2RAM74YLQBSNYSUOGCZFIGH2NKNUSBWHIRBIDKGNLFWPA, seg_pos={0 0}, upload=220186]: Uploading piece... Aug 17 13:04:38.194301585 (211h38m18.925569205s) [noise=true , address=45.91.116.7:4038 , piece_id=ZHCDUSOZWFP3PVAQCM5RH4QWLFBK4B64MVTOPN25IJWVLO6EN4XQ, seg_pos={0 0}, upload=220186]: Uploading piece... Aug 17 13:04:38.194509136 (211h38m18.925776785s) [noise=true , address=104.36.82.35:4614 , piece_id=ZOTYKJ46NWUGLBRM6MC7SM3EVJPNIYBXXQNAFB7N63G6CI2JLONA, seg_pos={0 0}, upload=220186]: Uploading piece... Aug 17 13:04:38.194546015 (211h38m18.925813635s) [noise=true , address=104.36.82.51:4079 , piece_id=LZGJRG7HCRAO33ZFPG4Y6CIEFVL6KMVAM6575LETYNME4SHOOGAA, seg_pos={0 0}, upload=220186]: Uploading piece... Aug 17 13:04:38.194806494 (211h38m18.926074114s) [noise=true , address=77.90.31.13:4643 , piece_id=GUI7DLZ7Z4BLGPAYH5JUWN7QF4KCG5AZNSMLMDAWAF3BJIOCHUQQ, seg_pos={0 0}, upload=220186]: Uploading piece... Aug 17 13:04:38.195064299 (211h38m18.926331918s) [noise=true , address=77.90.31.12:4618 , piece_id=OBMM7NZISIKW47JTDUT7ACXE4D2LLSFRGN5Y2ACHPE7LSI22MIQA, seg_pos={0 0}, upload=220186]: Uploading piece... Aug 17 13:04:38.195317073 (211h38m18.926584704s) [noise=true , address=77.90.31.19:4854 , piece_id=WODZKG5UCETL2RVVYFMOOFBLDWQNCPR472POA7AKGQDVIDPGWTWA, seg_pos={0 0}, upload=220186]: Uploading piece... Aug 17 13:04:38.195940586 (211h38m18.927208206s) [noise=true , address=104.36.82.19:4403 , piece_id=PZTCJ62URFB6HNZAF2TD2JQ4L6T6Y34PCXWKFWC356BT67EM7ZXA, seg_pos={0 0}, upload=220186]: Uploading piece... Aug 17 13:04:38.255222174 (211h38m18.986489804s) [upload=220186]: Next returned nil segment Aug 17 13:04:38.255236361 (211h38m18.986504s) [upload=220186]: Waiting for error group managing segments... Aug 17 13:04:38.279177878 (211h38m19.010445509s) [noise=true , address=45.91.116.7:4038 , piece_id=ZHCDUSOZWFP3PVAQCM5RH4QWLFBK4B64MVTOPN25IJWVLO6EN4XQ, seg_pos={0 0}, upload=220186]: Done uploading piece. err: <nil> Aug 17 13:04:38.298462160 (211h38m19.02972979s) [noise=true , address=77.90.31.19:4854 , piece_id=WODZKG5UCETL2RVVYFMOOFBLDWQNCPR472POA7AKGQDVIDPGWTWA, seg_pos={0 0}, upload=220186]: Done uploading piece. err: <nil> Aug 17 13:04:38.299709044 (211h38m19.030976694s) [noise=true , address=77.90.31.12:4618 , piece_id=OBMM7NZISIKW47JTDUT7ACXE4D2LLSFRGN5Y2ACHPE7LSI22MIQA, seg_pos={0 0}, upload=220186]: Done uploading piece. err: <nil> Aug 17 13:04:38.307320926 (211h38m19.038588556s) [noise=true , address=77.90.31.13:4643 , piece_id=GUI7DLZ7Z4BLGPAYH5JUWN7QF4KCG5AZNSMLMDAWAF3BJIOCHUQQ, seg_pos={0 0}, upload=220186]: Done uploading piece. err: <nil> Aug 17 13:04:38.319712446 (211h38m19.050980067s) [noise=true , address=45.91.116.10:4110 , piece_id=GIAG53YHED3CHN3GM37K2VMHSDO4CWKIHGX2R6E7WKTCX3SD6ICQ, seg_pos={0 0}, upload=220186]: Done uploading piece. err: <nil> Aug 17 13:04:38.401496927 (211h38m19.132764547s) [noise=true , address=79.127.220.101:4041 , piece_id=AIMGHT5ZZ6SU6MJDZ55T24K67MTU6LBPQC47QXODOEDRLSAXTSHA, seg_pos={0 0}, upload=220186]: Done uploading piece. err: <nil> Aug 17 13:04:38.570281095 (211h38m19.301548745s) [noise=true , address=77.90.31.15:4721 , piece_id=QLMQ4FXXQL4NIKNAPBBFG4QXVCTXPWZ3DZOKMNOKJ3UM3RSXCEZQ, seg_pos={0 0}, upload=220186]: Done uploading piece. err: <nil> Aug 17 13:04:38.794783452 (211h38m19.526051112s) [noise=true , address=66.23.193.175:4561 , piece_id=4RVKUSZPQ5HHQJG42N2K6X55F2FBB6UD6QBDJPTP6JKEYXKM3V3A, seg_pos={0 0}, upload=220186]: Done uploading piece. err: <nil> Aug 17 13:05:38.147857971 (211h39m18.879125601s) [noise=true , address=104.36.82.60:4353 , piece_id=337YPO4U6YEG5SYKOU3E5LGZMH56FQKW4S7WIZBLBPIEA3DH42KA, seg_pos={0 0}, upload=220186]: Done uploading piece. err: ecclient: upload failed (node:12BmzVLKbkqPMB7bJHGzFbgRRLu3TLg6TdTBCvYybsvz2yLpBbQ, address:104.36.82.60:4353): protocol: rpc: dial tcp 104.36.82.60:4353: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:38.147981312 (211h39m18.879248942s) [noise=true , address=104.36.82.51:4079 , piece_id=LZGJRG7HCRAO33ZFPG4Y6CIEFVL6KMVAM6575LETYNME4SHOOGAA, seg_pos={0 0}, upload=220186]: Done uploading piece. err: ecclient: upload failed (node:12RYKzn5zVSmxVbWwnBdX8oqtG1WE2MhPeEiWLf5dZeFAut6s2Y, address:104.36.82.51:4079): protocol: rpc: dial tcp 104.36.82.51:4079: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:38.148038480 (211h39m18.87930611s) [noise=true , address=104.36.82.19:4403 , piece_id=PZTCJ62URFB6HNZAF2TD2JQ4L6T6Y34PCXWKFWC356BT67EM7ZXA, seg_pos={0 0}, upload=220186]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:104.36.82.19:4403): protocol: rpc: dial tcp 104.36.82.19:4403: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:38.148089416 (211h39m18.879357066s) [noise=true , address=104.36.82.17:4331 , piece_id=7QWYB6C2RAM74YLQBSNYSUOGCZFIGH2NKNUSBWHIRBIDKGNLFWPA, seg_pos={0 0}, upload=220186]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:104.36.82.17:4331): protocol: rpc: dial tcp 104.36.82.17:4331: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:38.148170067 (211h39m18.879437697s) [noise=true , address=104.36.82.35:4614 , piece_id=ZOTYKJ46NWUGLBRM6MC7SM3EVJPNIYBXXQNAFB7N63G6CI2JLONA, seg_pos={0 0}, upload=220186]: Done uploading piece. err: ecclient: upload failed (node:12Pua5x8rAsuqYWCgoAY6fgy2SXNoDjzt7WwgMhzv1HN2KG5NY4, address:104.36.82.35:4614): protocol: write tcp 5.161.89.252:31456->104.36.82.35:4614: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:38.148184655 (211h39m18.879452315s) [noise=true , address=104.36.82.5:4849 , piece_id=CSB4SA3ENRUUCPNXHTDV6IQPURPAR4OC27XW4MZ4LX6RHSRQGA5Q, seg_pos={0 0}, upload=220186]: Done uploading piece. err: ecclient: upload failed (node:1n2gSzVmNCEZcfVsFKibjXHs2jMr82D3SWQEykFQQ4uyhhnrj1, address:104.36.82.5:4849): protocol: rpc: dial tcp 104.36.82.5:4849: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:38.148219901 (211h39m18.879487562s) [noise=true , address=104.36.82.23:4469 , piece_id=J6BIVVYKSYK4OHQZUA6COB2F7AARLIIKB6OUDEDGIOMP5YSWJZEA, seg_pos={0 0}, upload=220186]: Done uploading piece. err: ecclient: upload failed (node:1RWjSXyXqhgufuBewhCBgvBfkerSFeHxJZj88UzusCgiXvibQv, address:104.36.82.23:4469): protocol: rpc: dial tcp 104.36.82.23:4469: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:38.148266138 (211h39m18.879533768s) [noise=true , address=104.36.82.36:4626 , piece_id=53TRPZJPSOBJOYDCTJTEBO7IPYED2R57KDTXYW2KEFLCVT32HB7Q, seg_pos={0 0}, upload=220186]: Done uploading piece. err: ecclient: upload failed (node:12kx49CN28mLjnGhQBsdwH5zQyWypePDe3Ss15LyVEYmTczhHan, address:104.36.82.36:4626): protocol: rpc: dial tcp 104.36.82.36:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:38.148272991 (211h39m18.879540651s) [noise=true , address=104.36.82.59:4317 , piece_id=EOJBVJGNO4MJ6IRE7DGBH43UBWYOAV3L5LTKUXTFJGSHMZDWPLMA, seg_pos={0 0}, upload=220186]: Done uploading piece. err: ecclient: upload failed (node:12qjPfeVTW7hUd6VtE8XrhPPq2CCmYxZqxsDXps1CPGHpHahNEz, address:104.36.82.59:4317): protocol: rpc: dial tcp 104.36.82.59:4317: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:38.148287889 (211h39m18.879555539s) [noise=true , address=104.36.82.12:4169 , piece_id=E6Q46W63HR4LX4OVTT4RW5GGML42Q5EKX7PNNFPKIWMDWOUJITOQ, seg_pos={0 0}, upload=220186]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: write tcp 5.161.89.252:43340->104.36.82.12:4169: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:38.148314489 (211h39m18.879582149s) [seg_pos={0 0}, upload=220186]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Aug 17 13:05:38.161778888 (211h39m18.893046558s) [upload=220186]: Done uploading segments... Aug 17 13:05:38.161807702 (211h39m18.893075332s) [upload=220186]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 1m0.015834871s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration: 30.052479182s FAIL (2025-08-17 13:05:26 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 13:04:56.639707275 (211h38m37.370974895s) [upload=220192]: Starting upload Aug 17 13:04:56.639764082 (211h38m37.371031712s) [upload=220192]: Done starting upload Aug 17 13:04:56.639869460 (211h38m37.37113709s) [upload=220192]: Uploading segments... Aug 17 13:04:56.640069947 (211h38m37.371337567s) [upload=220192]: Got next segment. Inline: false Aug 17 13:04:56.640106225 (211h38m37.371373845s) [upload=220192]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:04:56.640110052 (211h38m37.371377683s) [upload=220192]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:04:56.679978849 (211h38m37.411246468s) [seg_pos={0 0}, upload=220192]: Begin upload segment... Aug 17 13:04:56.680065591 (211h38m37.411333211s) [seg_pos={0 0}, upload=220192]: Done begin upload segment. Aug 17 13:04:56.680086782 (211h38m37.411354412s) [noise=true , address=45.91.116.7:4010 , piece_id=DN7JRKNVCJLHRVGMCJLTARIM56EQ5GU4WQTWVGCTMY3S3GNMY56Q, seg_pos={0 0}, upload=220192]: Uploading piece... Aug 17 13:04:56.680104154 (211h38m37.411371764s) [noise=true , address=45.91.116.6:4975 , piece_id=X7TEJMCIMOKN2VFYMMZT2AQ2UXZCVBAOBYSTASZMWGDP2DYXHL3A, seg_pos={0 0}, upload=220192]: Uploading piece... Aug 17 13:04:56.680429245 (211h38m37.411696865s) [noise=true , address=104.36.82.6:4876 , piece_id=ODXAJUG4TL44IJL76RLV3AZF7HCITCOLTTGRF7XFWKVILHFNMVZQ, seg_pos={0 0}, upload=220192]: Uploading piece... Aug 17 13:04:56.680465142 (211h38m37.411732772s) [noise=true , address=104.36.82.4:4803 , piece_id=KGLEKMMAQDXR2WYCMFTL7AVB7ATTO7IT64V4HECJ7MZCXH3EN5LQ, seg_pos={0 0}, upload=220192]: Uploading piece... Aug 17 13:04:56.680626335 (211h38m37.411893956s) [noise=true , address=45.91.116.5:4299 , piece_id=57QQQUA2OVWTXLBFZGHOBFWWYGWQZBVGS3OBZZPYHUYC3RI6H5UA, seg_pos={0 0}, upload=220192]: Uploading piece... Aug 17 13:04:56.680757513 (211h38m37.412025143s) [noise=true , address=45.91.116.12:4227 , piece_id=4IH547A63T7UA3ZKSO7PSG7OBESMAFB6VFVJ24TJCDXFTEZ766HA, seg_pos={0 0}, upload=220192]: Uploading piece... Aug 17 13:04:56.680856739 (211h38m37.412124389s) [noise=true , address=104.36.82.2:4732 , piece_id=4HDVY5GL757DOR4NYVUPQHSXLNCIOQODTQNJZIPP36RY46CY6HYQ, seg_pos={0 0}, upload=220192]: Uploading piece... Aug 17 13:04:56.681002013 (211h38m37.412269643s) [noise=true , address=45.91.116.8:4045 , piece_id=N6ZBLRCQUNRYIKSUTOH53Z4XGJTP6M45YUVKTFOID2B4MFSZ3TQA, seg_pos={0 0}, upload=220192]: Uploading piece... Aug 17 13:04:56.681059570 (211h38m37.412327191s) [noise=true , address=104.36.82.5:4840 , piece_id=3MLUZQ7YCLEFSJ7Q2OS3GPHC4IDHS4PUOMEI4UAFWGFBLC2IWQ2A, seg_pos={0 0}, upload=220192]: Uploading piece... Aug 17 13:04:56.681269235 (211h38m37.412536854s) [noise=true , address=104.36.82.3:4767 , piece_id=XCM475VIQURKLGVHPYV227ZXF5TLAEL3MS36RQKETISG234ZK6PQ, seg_pos={0 0}, upload=220192]: Uploading piece... Aug 17 13:04:56.681289332 (211h38m37.412556952s) [noise=true , address=45.91.116.13:4155 , piece_id=CAUZBPFOHWKORYSTQGFHWS3TGB3CEOGQRQC4L5QK2PADXT7APELA, seg_pos={0 0}, upload=220192]: Uploading piece... Aug 17 13:04:56.681468479 (211h38m37.412736099s) [noise=true , address=104.36.82.10:4079 , piece_id=R2HPIMAGTVBICCKYAJFU5Y4IEYJYKTVLUICW77WGJ2IUQWXDKU5A, seg_pos={0 0}, upload=220192]: Uploading piece... Aug 17 13:04:56.681513535 (211h38m37.412781154s) [noise=true , address=45.91.116.4:4937 , piece_id=ZQCVMZWONQLPW4EMZ7GVNJ7P7WOR6PW75SU4KZD7KUBUMHSZKHTQ, seg_pos={0 0}, upload=220192]: Uploading piece... Aug 17 13:04:56.681635644 (211h38m37.412903254s) [noise=true , address=45.91.116.14:4191 , piece_id=BAATG657WKMRQBUNEJASE7URMMO4UWGURAUUOP3ZG6GD34C33E2A, seg_pos={0 0}, upload=220192]: Uploading piece... Aug 17 13:04:56.681771359 (211h38m37.413038979s) [noise=true , address=45.91.116.9:4263 , piece_id=UNEDLF3WFZ6L5V4YSHXFHX554C2JJVU72IIQVOJPTM3NSYQX7S5A, seg_pos={0 0}, upload=220192]: Uploading piece... Aug 17 13:04:56.682097753 (211h38m37.413365372s) [noise=true , address=77.90.31.15:4699 , piece_id=BWRDI5V3HGDL6RMOLXWO7AJHI6BFHGDD3ZH45LPMUXRF3HTINOQQ, seg_pos={0 0}, upload=220192]: Uploading piece... Aug 17 13:04:56.755359417 (211h38m37.486627038s) [upload=220192]: Next returned nil segment Aug 17 13:04:56.755370489 (211h38m37.486638119s) [upload=220192]: Waiting for error group managing segments... Aug 17 13:04:56.763717184 (211h38m37.494984814s) [noise=true , address=45.91.116.5:4299 , piece_id=57QQQUA2OVWTXLBFZGHOBFWWYGWQZBVGS3OBZZPYHUYC3RI6H5UA, seg_pos={0 0}, upload=220192]: Done uploading piece. err: <nil> Aug 17 13:04:56.764096316 (211h38m37.495363936s) [noise=true , address=45.91.116.13:4155 , piece_id=CAUZBPFOHWKORYSTQGFHWS3TGB3CEOGQRQC4L5QK2PADXT7APELA, seg_pos={0 0}, upload=220192]: Done uploading piece. err: <nil> Aug 17 13:04:56.764573083 (211h38m37.495840713s) [noise=true , address=45.91.116.12:4227 , piece_id=4IH547A63T7UA3ZKSO7PSG7OBESMAFB6VFVJ24TJCDXFTEZ766HA, seg_pos={0 0}, upload=220192]: Done uploading piece. err: <nil> Aug 17 13:04:56.764894978 (211h38m37.496162598s) [noise=true , address=45.91.116.4:4937 , piece_id=ZQCVMZWONQLPW4EMZ7GVNJ7P7WOR6PW75SU4KZD7KUBUMHSZKHTQ, seg_pos={0 0}, upload=220192]: Done uploading piece. err: <nil> Aug 17 13:04:56.765285482 (211h38m37.496553112s) [noise=true , address=45.91.116.6:4975 , piece_id=X7TEJMCIMOKN2VFYMMZT2AQ2UXZCVBAOBYSTASZMWGDP2DYXHL3A, seg_pos={0 0}, upload=220192]: Done uploading piece. err: <nil> Aug 17 13:04:56.766276976 (211h38m37.497544607s) [noise=true , address=45.91.116.7:4010 , piece_id=DN7JRKNVCJLHRVGMCJLTARIM56EQ5GU4WQTWVGCTMY3S3GNMY56Q, seg_pos={0 0}, upload=220192]: Done uploading piece. err: <nil> Aug 17 13:04:56.766644017 (211h38m37.497911647s) [noise=true , address=45.91.116.9:4263 , piece_id=UNEDLF3WFZ6L5V4YSHXFHX554C2JJVU72IIQVOJPTM3NSYQX7S5A, seg_pos={0 0}, upload=220192]: Done uploading piece. err: <nil> Aug 17 13:04:56.783429178 (211h38m37.514696808s) [noise=true , address=45.91.116.14:4191 , piece_id=BAATG657WKMRQBUNEJASE7URMMO4UWGURAUUOP3ZG6GD34C33E2A, seg_pos={0 0}, upload=220192]: Done uploading piece. err: <nil> Aug 17 13:04:56.790288245 (211h38m37.521555865s) [noise=true , address=45.91.116.8:4045 , piece_id=N6ZBLRCQUNRYIKSUTOH53Z4XGJTP6M45YUVKTFOID2B4MFSZ3TQA, seg_pos={0 0}, upload=220192]: Done uploading piece. err: <nil> Aug 17 13:04:56.801159878 (211h38m37.532427508s) [noise=true , address=77.90.31.15:4699 , piece_id=BWRDI5V3HGDL6RMOLXWO7AJHI6BFHGDD3ZH45LPMUXRF3HTINOQQ, seg_pos={0 0}, upload=220192]: Done uploading piece. err: <nil> Aug 17 13:05:26.622086868 (211h39m7.353354488s) [noise=true , address=104.36.82.4:4803 , piece_id=KGLEKMMAQDXR2WYCMFTL7AVB7ATTO7IT64V4HECJ7MZCXH3EN5LQ, seg_pos={0 0}, upload=220192]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: rpc: dial tcp 104.36.82.4:4803: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:26.622128657 (211h39m7.353396287s) [noise=true , address=104.36.82.2:4732 , piece_id=4HDVY5GL757DOR4NYVUPQHSXLNCIOQODTQNJZIPP36RY46CY6HYQ, seg_pos={0 0}, upload=220192]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:26.622258390 (211h39m7.35352602s) [noise=true , address=104.36.82.5:4840 , piece_id=3MLUZQ7YCLEFSJ7Q2OS3GPHC4IDHS4PUOMEI4UAFWGFBLC2IWQ2A, seg_pos={0 0}, upload=220192]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:104.36.82.5:4840): protocol: rpc: dial tcp 104.36.82.5:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:26.622269691 (211h39m7.353537342s) [noise=true , address=104.36.82.6:4876 , piece_id=ODXAJUG4TL44IJL76RLV3AZF7HCITCOLTTGRF7XFWKVILHFNMVZQ, seg_pos={0 0}, upload=220192]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: rpc: dial tcp 104.36.82.6:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:26.622300420 (211h39m7.35356805s) [noise=true , address=104.36.82.10:4079 , piece_id=R2HPIMAGTVBICCKYAJFU5Y4IEYJYKTVLUICW77WGJ2IUQWXDKU5A, seg_pos={0 0}, upload=220192]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: write tcp 5.161.89.252:49684->104.36.82.10:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:26.622328412 (211h39m7.353596042s) [noise=true , address=104.36.82.3:4767 , piece_id=XCM475VIQURKLGVHPYV227ZXF5TLAEL3MS36RQKETISG234ZK6PQ, seg_pos={0 0}, upload=220192]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: rpc: dial tcp 104.36.82.3:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:26.622360823 (211h39m7.353628453s) [seg_pos={0 0}, upload=220192]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 13:05:26.673123193 (211h39m7.404390843s) [upload=220192]: Done uploading segments... Aug 17 13:05:26.673147128 (211h39m7.404414778s) [upload=220192]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.052498609s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [....X................................................X...XXX] duration: 1m0.022307943s FAIL (2025-08-17 13:05:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== Aug 17 13:04:03.070845393 (211h37m43.802113033s) [upload=220178]: Starting upload Aug 17 13:04:03.070923189 (211h37m43.80219084s) [upload=220178]: Done starting upload Aug 17 13:04:03.070984785 (211h37m43.802252426s) [upload=220178]: Uploading segments... Aug 17 13:04:03.071334002 (211h37m43.802601652s) [upload=220178]: Got next segment. Inline: false Aug 17 13:04:03.071404434 (211h37m43.802672074s) [upload=220178]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:04:03.071410826 (211h37m43.802678476s) [upload=220178]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:04:03.096188275 (211h37m43.827455896s) [seg_pos={0 0}, upload=220178]: Begin upload segment... Aug 17 13:04:03.096243459 (211h37m43.827511079s) [seg_pos={0 0}, upload=220178]: Done begin upload segment. Aug 17 13:04:03.096264739 (211h37m43.827532359s) [noise=true , address=45.91.116.11:4146 , piece_id=Z5M2IIHQWUD5AASRAVOLWQ5CFIL4OIJQCU4EZAFHBZ4T3BVSRWHQ, seg_pos={0 0}, upload=220178]: Uploading piece... Aug 17 13:04:03.096297090 (211h37m43.827564711s) [noise=true , address=104.36.82.8:4004 , piece_id=SAM4DMSPWSBCZ37TNZVRGPMF6LWJEZBHIL3BPKXEZ3KEYP7BGUGQ, seg_pos={0 0}, upload=220178]: Uploading piece... Aug 17 13:04:03.096561517 (211h37m43.827829147s) [noise=true , address=77.90.31.19:4876 , piece_id=TRGB325BOWLPNMC2AFD7HTLB6JFQOH4XLIFTKTW4ZJRS377LFZIA, seg_pos={0 0}, upload=220178]: Uploading piece... Aug 17 13:04:03.096609257 (211h37m43.827876867s) [noise=true , address=104.36.82.10:4105 , piece_id=7ZKVNDV475EFN2SRIBJ3VM3LZ6TKJE2BQJLXAS5I67KPIBSTVUFA, seg_pos={0 0}, upload=220178]: Uploading piece... Aug 17 13:04:03.096785047 (211h37m43.828052678s) [noise=true , address=77.90.31.13:4640 , piece_id=HGPFMUS3GQMEXY6YGRL2RS6F4WIGHS7X6SFURZKPGUBXGOSRSJAQ, seg_pos={0 0}, upload=220178]: Uploading piece... Aug 17 13:04:03.096836193 (211h37m43.828103813s) [noise=true , address=66.23.193.175:4560 , piece_id=ZVJGY7JVZAMRJJWLOFFBLBJ675YJMISNHHINQBGMF2LXXNTANPFQ, seg_pos={0 0}, upload=220178]: Uploading piece... Aug 17 13:04:03.097041099 (211h37m43.828308709s) [noise=true , address=77.90.31.10:4552 , piece_id=EAP5CJ74FASVWAXNW5LKDRLBBTDZUTDS7UORLMUX7OIJCEMRO5GA, seg_pos={0 0}, upload=220178]: Uploading piece... Aug 17 13:04:03.097069563 (211h37m43.828337193s) [noise=true , address=104.36.82.60:4358 , piece_id=AYT5BJD3ALETPMOFCMDVEQVDRO23EHDFHYUFY2FCHDGLOY4TUYTA, seg_pos={0 0}, upload=220178]: Uploading piece... Aug 17 13:04:03.097282993 (211h37m43.828550613s) [noise=true , address=45.91.116.4:4971 , piece_id=U4G7YWWRQOTKJGZAVAPTF25RQRBG6SE4EWG6PKGI7DLMYT7DKQRQ, seg_pos={0 0}, upload=220178]: Uploading piece... Aug 17 13:04:03.097458905 (211h37m43.828726544s) [noise=true , address=104.36.82.23:4462 , piece_id=SXNRXVPHL4LOELLQW4GORETTHHQJUZLMDTJMCBS4AHCZNMPKTRRA, seg_pos={0 0}, upload=220178]: Uploading piece... Aug 17 13:04:03.097482419 (211h37m43.828750028s) [noise=true , address=104.36.82.59:4305 , piece_id=ED5EKQLDCYLQR6JNYTOKI47SB67JJHXRJUR7LGBSF6QA47MLE4KQ, seg_pos={0 0}, upload=220178]: Uploading piece... Aug 17 13:04:03.097709105 (211h37m43.828976725s) [noise=true , address=104.36.82.5:4856 , piece_id=GK7RI6Y555K3GPPMDE2D33XJBRX4NKIHU7TUJ4WOAHKEOJQ3I7TQ, seg_pos={0 0}, upload=220178]: Uploading piece... Aug 17 13:04:03.097751906 (211h37m43.829019526s) [noise=true , address=104.36.82.58:4286 , piece_id=HCF7LTJPDK5FVIFZVSLAFUGBXSYRZGY6746NEIWSKANOPONH6TXA, seg_pos={0 0}, upload=220178]: Uploading piece... Aug 17 13:04:03.097954878 (211h37m43.829222498s) [noise=true , address=104.36.82.17:4362 , piece_id=FPJMTII3LYFO2KTPSSLZUWS7NQV3TBOCY5DYULRZ2FHQY4FGCWIA, seg_pos={0 0}, upload=220178]: Uploading piece... Aug 17 13:04:03.098008538 (211h37m43.829276159s) [noise=true , address=104.36.82.2:4734 , piece_id=YLQTEJLEZO2ESBTWTBM7GR4IPSF5KEH3LQ6QO7P2T4DD75KZ3LCQ, seg_pos={0 0}, upload=220178]: Uploading piece... Aug 17 13:04:03.098284467 (211h37m43.829552097s) [noise=true , address=79.127.205.246:4267 , piece_id=MN35PA4IOCWHKYG2E26HUKDFYQNSFVG4U4PNYSIFKQJS6V64K2LA, seg_pos={0 0}, upload=220178]: Uploading piece... Aug 17 13:04:03.098516102 (211h37m43.829783732s) [noise=true , address=79.127.201.215:4346 , piece_id=3W5XJCMI5FFFF624MRCTCUEGS3L2ODW67FGE5WYDLANKY72BKKVQ, seg_pos={0 0}, upload=220178]: Uploading piece... Aug 17 13:04:03.098748779 (211h37m43.8300164s) [noise=true , address=104.36.82.15:4288 , piece_id=Y2JN6PFCGPF6LZJJ22YCIW6GRLE3VR7JCP7AKZ6IYWWR5CG47WHA, seg_pos={0 0}, upload=220178]: Uploading piece... Aug 17 13:04:03.162307852 (211h37m43.893575482s) [upload=220178]: Next returned nil segment Aug 17 13:04:03.162324893 (211h37m43.893592523s) [upload=220178]: Waiting for error group managing segments... Aug 17 13:04:03.191470276 (211h37m43.922737896s) [noise=true , address=45.91.116.4:4971 , piece_id=U4G7YWWRQOTKJGZAVAPTF25RQRBG6SE4EWG6PKGI7DLMYT7DKQRQ, seg_pos={0 0}, upload=220178]: Done uploading piece. err: <nil> Aug 17 13:04:03.206121706 (211h37m43.937389326s) [noise=true , address=45.91.116.11:4146 , piece_id=Z5M2IIHQWUD5AASRAVOLWQ5CFIL4OIJQCU4EZAFHBZ4T3BVSRWHQ, seg_pos={0 0}, upload=220178]: Done uploading piece. err: <nil> Aug 17 13:04:03.206734948 (211h37m43.938002598s) [noise=true , address=77.90.31.19:4876 , piece_id=TRGB325BOWLPNMC2AFD7HTLB6JFQOH4XLIFTKTW4ZJRS377LFZIA, seg_pos={0 0}, upload=220178]: Done uploading piece. err: <nil> Aug 17 13:04:03.215645653 (211h37m43.946913303s) [noise=true , address=77.90.31.10:4552 , piece_id=EAP5CJ74FASVWAXNW5LKDRLBBTDZUTDS7UORLMUX7OIJCEMRO5GA, seg_pos={0 0}, upload=220178]: Done uploading piece. err: <nil> Aug 17 13:04:03.294254753 (211h37m44.025522373s) [noise=true , address=77.90.31.13:4640 , piece_id=HGPFMUS3GQMEXY6YGRL2RS6F4WIGHS7X6SFURZKPGUBXGOSRSJAQ, seg_pos={0 0}, upload=220178]: Done uploading piece. err: <nil> Aug 17 13:04:03.406642228 (211h37m44.137909878s) [noise=true , address=79.127.201.215:4346 , piece_id=3W5XJCMI5FFFF624MRCTCUEGS3L2ODW67FGE5WYDLANKY72BKKVQ, seg_pos={0 0}, upload=220178]: Done uploading piece. err: <nil> Aug 17 13:04:03.583439103 (211h37m44.314706733s) [noise=true , address=79.127.205.246:4267 , piece_id=MN35PA4IOCWHKYG2E26HUKDFYQNSFVG4U4PNYSIFKQJS6V64K2LA, seg_pos={0 0}, upload=220178]: Done uploading piece. err: <nil> Aug 17 13:04:03.695284547 (211h37m44.426552187s) [noise=true , address=66.23.193.175:4560 , piece_id=ZVJGY7JVZAMRJJWLOFFBLBJ675YJMISNHHINQBGMF2LXXNTANPFQ, seg_pos={0 0}, upload=220178]: Done uploading piece. err: <nil> Aug 17 13:05:03.058245439 (211h38m43.789513069s) [noise=true , address=104.36.82.58:4286 , piece_id=HCF7LTJPDK5FVIFZVSLAFUGBXSYRZGY6746NEIWSKANOPONH6TXA, seg_pos={0 0}, upload=220178]: Done uploading piece. err: ecclient: upload failed (node:12v2vJnE92VrJXQrbxnAxANHCVASuXiBKMUZECMVmEWEjsZRtsQ, address:104.36.82.58:4286): protocol: rpc: dial tcp 104.36.82.58:4286: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:03.058296676 (211h38m43.789564306s) [noise=true , address=104.36.82.59:4305 , piece_id=ED5EKQLDCYLQR6JNYTOKI47SB67JJHXRJUR7LGBSF6QA47MLE4KQ, seg_pos={0 0}, upload=220178]: Done uploading piece. err: ecclient: upload failed (node:122nFQhbiCN9A3p3qQFsWKwXFFcmtWRXDLBAP57rcLcQBVhShHB, address:104.36.82.59:4305): protocol: rpc: dial tcp 104.36.82.59:4305: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:03.058327133 (211h38m43.789594763s) [noise=true , address=104.36.82.5:4856 , piece_id=GK7RI6Y555K3GPPMDE2D33XJBRX4NKIHU7TUJ4WOAHKEOJQ3I7TQ, seg_pos={0 0}, upload=220178]: Done uploading piece. err: ecclient: upload failed (node:12mXTGa9rxx8jU8oKLmR9obX3Utn7p5rQH2GeK6LeJ6jiNvfYZ6, address:104.36.82.5:4856): protocol: rpc: dial tcp 104.36.82.5:4856: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:03.058349134 (211h38m43.789616754s) [noise=true , address=104.36.82.60:4358 , piece_id=AYT5BJD3ALETPMOFCMDVEQVDRO23EHDFHYUFY2FCHDGLOY4TUYTA, seg_pos={0 0}, upload=220178]: Done uploading piece. err: ecclient: upload failed (node:1dwCNEcA9o51ZN3UaN3Frxdawwf8DXBj86iQMywjW8XP4ESmw1, address:104.36.82.60:4358): protocol: rpc: dial tcp 104.36.82.60:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:03.058408897 (211h38m43.789676517s) [noise=true , address=104.36.82.15:4288 , piece_id=Y2JN6PFCGPF6LZJJ22YCIW6GRLE3VR7JCP7AKZ6IYWWR5CG47WHA, seg_pos={0 0}, upload=220178]: Done uploading piece. err: ecclient: upload failed (node:1gPjPGuueZnizLT2QvJN9y7riwKiW5TWMkQtmGGPP7VZvkNoB7, address:104.36.82.15:4288): protocol: rpc: dial tcp 104.36.82.15:4288: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:03.058445856 (211h38m43.789713476s) [noise=true , address=104.36.82.17:4362 , piece_id=FPJMTII3LYFO2KTPSSLZUWS7NQV3TBOCY5DYULRZ2FHQY4FGCWIA, seg_pos={0 0}, upload=220178]: Done uploading piece. err: ecclient: upload failed (node:123zeRoJ5g7juzsV7qQeNacC5BB7jZpfDBnznFhySGWfyPGJ4oa, address:104.36.82.17:4362): protocol: rpc: dial tcp 104.36.82.17:4362: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:03.058475762 (211h38m43.789743392s) [noise=true , address=104.36.82.2:4734 , piece_id=YLQTEJLEZO2ESBTWTBM7GR4IPSF5KEH3LQ6QO7P2T4DD75KZ3LCQ, seg_pos={0 0}, upload=220178]: Done uploading piece. err: ecclient: upload failed (node:12GLb5LQje9wwivnj8Wj1tXcnS6mFb2Bco5saNwgmFJBVDujrv7, address:104.36.82.2:4734): protocol: rpc: dial tcp 104.36.82.2:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:03.058515928 (211h38m43.789783558s) [noise=true , address=104.36.82.23:4462 , piece_id=SXNRXVPHL4LOELLQW4GORETTHHQJUZLMDTJMCBS4AHCZNMPKTRRA, seg_pos={0 0}, upload=220178]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: write tcp 5.161.89.252:50604->104.36.82.23:4462: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:03.058521287 (211h38m43.789788918s) [noise=true , address=104.36.82.8:4004 , piece_id=SAM4DMSPWSBCZ37TNZVRGPMF6LWJEZBHIL3BPKXEZ3KEYP7BGUGQ, seg_pos={0 0}, upload=220178]: Done uploading piece. err: ecclient: upload failed (node:12BdVmmJ1L1sEMcHXTAdXBXU6dmdhUE9VT74jCNdZj5bMqJWxqp, address:104.36.82.8:4004): protocol: write tcp 5.161.89.252:18160->104.36.82.8:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:03.058525596 (211h38m43.789793216s) [noise=true , address=104.36.82.10:4105 , piece_id=7ZKVNDV475EFN2SRIBJ3VM3LZ6TKJE2BQJLXAS5I67KPIBSTVUFA, seg_pos={0 0}, upload=220178]: Done uploading piece. err: ecclient: upload failed (node:12GhDrKMcDEg2RxamqKtiHphpgNdvtZYdHXrjzGsNJFjBFYeEen, address:104.36.82.10:4105): protocol: rpc: dial tcp 104.36.82.10:4105: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:03.058552196 (211h38m43.789819815s) [seg_pos={0 0}, upload=220178]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Aug 17 13:05:03.079383485 (211h38m43.810651105s) [upload=220178]: Done uploading segments... Aug 17 13:05:03.079402460 (211h38m43.810670081s) [upload=220178]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 1m0.022319986s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X..............X.....] duration: 564.61665ms FAIL (2025-08-17 13:04:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 564.639822ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................................XXXX] duration: 30.019739699s FAIL (2025-08-17 13:04:26 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Aug 17 13:03:56.549192804 (211h37m37.280460454s) [upload=220174]: Starting upload Aug 17 13:03:56.549322689 (211h37m37.280590338s) [upload=220174]: Done starting upload Aug 17 13:03:56.549418540 (211h37m37.28068617s) [upload=220174]: Uploading segments... Aug 17 13:03:56.549694478 (211h37m37.280962128s) [upload=220174]: Got next segment. Inline: false Aug 17 13:03:56.549734694 (211h37m37.281002343s) [upload=220174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:03:56.549740074 (211h37m37.281007723s) [upload=220174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:03:56.574190278 (211h37m37.305457908s) [seg_pos={0 0}, upload=220174]: Begin upload segment... Aug 17 13:03:56.574250441 (211h37m37.305518061s) [seg_pos={0 0}, upload=220174]: Done begin upload segment. Aug 17 13:03:56.574273784 (211h37m37.305541404s) [noise=true , address=45.91.116.6:4974 , piece_id=J3LOEYSM6EZ5YCO3LVXYLEX2EPBWXWJGY54XSK6LY5M3MTTWMDMA, seg_pos={0 0}, upload=220174]: Uploading piece... Aug 17 13:03:56.574341742 (211h37m37.305609402s) [noise=true , address=77.90.31.13:4628 , piece_id=5OSR6YLJCS2HE7VG7T5UXH5TDOUGHFCNSKT7IXCG4LI4C26TORFA, seg_pos={0 0}, upload=220174]: Uploading piece... Aug 17 13:03:56.574582696 (211h37m37.305850316s) [noise=true , address=77.90.31.12:4589 , piece_id=RW7XBAZTHIL77G3U5H5PCOT3UC7EESUSWYP2GOD2PF2PSQJJTZLQ, seg_pos={0 0}, upload=220174]: Uploading piece... Aug 17 13:03:56.574816956 (211h37m37.306084576s) [noise=true , address=45.91.116.5:4297 , piece_id=YPWGBA4HQCZTNMFCFYM5NNQCBQ7GUCDQHES63IQV3VNTFJI5XUMQ, seg_pos={0 0}, upload=220174]: Uploading piece... Aug 17 13:03:56.575032912 (211h37m37.306300542s) [noise=true , address=45.91.116.6:4975 , piece_id=VPIA5T5FOL5REMO6E2GVRMDQTHRUFN3FI7UTBIGUM4VXKJUOH24A, seg_pos={0 0}, upload=220174]: Uploading piece... Aug 17 13:03:56.575082355 (211h37m37.306350015s) [noise=true , address=104.36.82.3:4765 , piece_id=UWCR5Z4UEGCBAK6K2BLMXEAGDCEIIGHLQ3YIYMJMOE5KAPMYSIYA, seg_pos={0 0}, upload=220174]: Uploading piece... Aug 17 13:03:56.575259418 (211h37m37.306527048s) [noise=true , address=45.91.116.9:4263 , piece_id=IKEH6IBSBOTZTUY6GIRFFD5MV3EEGOB2IY4TQQOMMHOUY2SNHFMQ, seg_pos={0 0}, upload=220174]: Uploading piece... Aug 17 13:03:56.575469814 (211h37m37.306737434s) [noise=true , address=45.91.116.7:4009 , piece_id=T7H3ARWU4YCHWX3DZNHA25JF64I7CAZT2PB7YTZ2AEOR74ZSW4XA, seg_pos={0 0}, upload=220174]: Uploading piece... Aug 17 13:03:56.575681091 (211h37m37.306948741s) [noise=true , address=104.36.82.5:4840 , piece_id=5BCU3AQPB73RPO72JNSUU22Y3X3J2SVMIFUIPZGDSMYPV46XZ5CA, seg_pos={0 0}, upload=220174]: Uploading piece... Aug 17 13:03:56.575694366 (211h37m37.306961996s) [noise=true , address=45.91.116.10:4082 , piece_id=HUVP2XIWL232UYENWOGP6HU44C6HL527BDJ25RETCAKRS6HVSPJA, seg_pos={0 0}, upload=220174]: Uploading piece... Aug 17 13:03:56.576171122 (211h37m37.307438772s) [noise=true , address=45.91.116.14:4190 , piece_id=CQQ6O6PXLHB2BQVZNXHS4AAWPZ7N6KXDPZXCSDDHXNJOEDXIW2EQ, seg_pos={0 0}, upload=220174]: Uploading piece... Aug 17 13:03:56.576728640 (211h37m37.307996291s) [noise=true , address=104.36.82.13:4185 , piece_id=AJYG36NWJQFR57NRK4WEN27SBQLI2NJFBWFUUKXHXP7UYCHCM3HA, seg_pos={0 0}, upload=220174]: Uploading piece... Aug 17 13:03:56.577346092 (211h37m37.308613742s) [noise=true , address=45.91.116.12:4226 , piece_id=N5VHTPIRNPTX7VY4Q6PWRHAW2PB6UWOWXTCUFI55OWL6LU6AOKMQ, seg_pos={0 0}, upload=220174]: Uploading piece... Aug 17 13:03:56.578067749 (211h37m37.309335399s) [noise=true , address=104.36.82.14:4223 , piece_id=I24LHHGEKW2BECT6F2YKCACUMSZYHMVDOGEBP53TLCFVOEL6OJBQ, seg_pos={0 0}, upload=220174]: Uploading piece... Aug 17 13:03:56.582124006 (211h37m37.313391655s) [noise=true , address=77.90.31.18:4807 , piece_id=THMR3SGEEJUJG2MUNRWJFKKJD6KHAA3G3RWC2BRON3L6ENKHM7XA, seg_pos={0 0}, upload=220174]: Uploading piece... Aug 17 13:03:56.594785163 (211h37m37.326052813s) [noise=true , address=104.36.82.2:4729 , piece_id=Q56OQARCE5BZDYP5IQJOHC2RJEGKZQMLHHBMJC6VPJI43XJOFPHA, seg_pos={0 0}, upload=220174]: Uploading piece... Aug 17 13:03:56.656962206 (211h37m37.388229846s) [upload=220174]: Next returned nil segment Aug 17 13:03:56.656975230 (211h37m37.38824286s) [upload=220174]: Waiting for error group managing segments... Aug 17 13:03:56.666208090 (211h37m37.397475731s) [noise=true , address=45.91.116.7:4009 , piece_id=T7H3ARWU4YCHWX3DZNHA25JF64I7CAZT2PB7YTZ2AEOR74ZSW4XA, seg_pos={0 0}, upload=220174]: Done uploading piece. err: <nil> Aug 17 13:03:56.666508346 (211h37m37.397775965s) [noise=true , address=45.91.116.5:4297 , piece_id=YPWGBA4HQCZTNMFCFYM5NNQCBQ7GUCDQHES63IQV3VNTFJI5XUMQ, seg_pos={0 0}, upload=220174]: Done uploading piece. err: <nil> Aug 17 13:03:56.666844567 (211h37m37.398112187s) [noise=true , address=45.91.116.9:4263 , piece_id=IKEH6IBSBOTZTUY6GIRFFD5MV3EEGOB2IY4TQQOMMHOUY2SNHFMQ, seg_pos={0 0}, upload=220174]: Done uploading piece. err: <nil> Aug 17 13:03:56.666939826 (211h37m37.398207467s) [noise=true , address=45.91.116.10:4082 , piece_id=HUVP2XIWL232UYENWOGP6HU44C6HL527BDJ25RETCAKRS6HVSPJA, seg_pos={0 0}, upload=220174]: Done uploading piece. err: <nil> Aug 17 13:03:56.667128472 (211h37m37.398396091s) [noise=true , address=45.91.116.6:4974 , piece_id=J3LOEYSM6EZ5YCO3LVXYLEX2EPBWXWJGY54XSK6LY5M3MTTWMDMA, seg_pos={0 0}, upload=220174]: Done uploading piece. err: <nil> Aug 17 13:03:56.667812988 (211h37m37.399080629s) [noise=true , address=45.91.116.6:4975 , piece_id=VPIA5T5FOL5REMO6E2GVRMDQTHRUFN3FI7UTBIGUM4VXKJUOH24A, seg_pos={0 0}, upload=220174]: Done uploading piece. err: <nil> Aug 17 13:03:56.669064491 (211h37m37.400332112s) [noise=true , address=45.91.116.14:4190 , piece_id=CQQ6O6PXLHB2BQVZNXHS4AAWPZ7N6KXDPZXCSDDHXNJOEDXIW2EQ, seg_pos={0 0}, upload=220174]: Done uploading piece. err: <nil> Aug 17 13:03:56.674753349 (211h37m37.406020999s) [noise=true , address=77.90.31.18:4807 , piece_id=THMR3SGEEJUJG2MUNRWJFKKJD6KHAA3G3RWC2BRON3L6ENKHM7XA, seg_pos={0 0}, upload=220174]: Done uploading piece. err: <nil> Aug 17 13:03:56.677108467 (211h37m37.408376087s) [noise=true , address=45.91.116.12:4226 , piece_id=N5VHTPIRNPTX7VY4Q6PWRHAW2PB6UWOWXTCUFI55OWL6LU6AOKMQ, seg_pos={0 0}, upload=220174]: Done uploading piece. err: <nil> Aug 17 13:03:56.700591011 (211h37m37.431858662s) [noise=true , address=77.90.31.12:4589 , piece_id=RW7XBAZTHIL77G3U5H5PCOT3UC7EESUSWYP2GOD2PF2PSQJJTZLQ, seg_pos={0 0}, upload=220174]: Done uploading piece. err: <nil> Aug 17 13:03:56.708268738 (211h37m37.439536368s) [noise=true , address=77.90.31.13:4628 , piece_id=5OSR6YLJCS2HE7VG7T5UXH5TDOUGHFCNSKT7IXCG4LI4C26TORFA, seg_pos={0 0}, upload=220174]: Done uploading piece. err: <nil> Aug 17 13:04:26.534712308 (211h38m7.265979958s) [noise=true , address=104.36.82.14:4223 , piece_id=I24LHHGEKW2BECT6F2YKCACUMSZYHMVDOGEBP53TLCFVOEL6OJBQ, seg_pos={0 0}, upload=220174]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 5.161.89.252:34198->104.36.82.14:4223: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:04:26.535137728 (211h38m7.266405388s) [noise=true , address=104.36.82.2:4729 , piece_id=Q56OQARCE5BZDYP5IQJOHC2RJEGKZQMLHHBMJC6VPJI43XJOFPHA, seg_pos={0 0}, upload=220174]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: write tcp 5.161.89.252:42838->104.36.82.2:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:04:26.535173806 (211h38m7.266441456s) [noise=true , address=104.36.82.13:4185 , piece_id=AJYG36NWJQFR57NRK4WEN27SBQLI2NJFBWFUUKXHXP7UYCHCM3HA, seg_pos={0 0}, upload=220174]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:104.36.82.13:4185): protocol: rpc: dial tcp 104.36.82.13:4185: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:04:26.535257704 (211h38m7.266525355s) [noise=true , address=104.36.82.3:4765 , piece_id=UWCR5Z4UEGCBAK6K2BLMXEAGDCEIIGHLQ3YIYMJMOE5KAPMYSIYA, seg_pos={0 0}, upload=220174]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:104.36.82.3:4765): protocol: write tcp 5.161.89.252:51786->104.36.82.3:4765: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:04:26.535290466 (211h38m7.266558116s) [noise=true , address=104.36.82.5:4840 , piece_id=5BCU3AQPB73RPO72JNSUU22Y3X3J2SVMIFUIPZGDSMYPV46XZ5CA, seg_pos={0 0}, upload=220174]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:104.36.82.5:4840): protocol: write tcp 5.161.89.252:8162->104.36.82.5:4840: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:04:26.535385744 (211h38m7.266653405s) [seg_pos={0 0}, upload=220174]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Aug 17 13:04:26.553247219 (211h38m7.28451484s) [upload=220174]: Done uploading segments... Aug 17 13:04:26.553269472 (211h38m7.284537102s) [upload=220174]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.019763935s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....X................................................X...XX] duration: 1m0.017881992s FAIL (2025-08-17 13:04:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 9) ======LOGS====== Aug 17 13:03:02.981068696 (211h36m43.712336326s) [upload=220158]: Starting upload Aug 17 13:03:02.981111587 (211h36m43.712379217s) [upload=220158]: Done starting upload Aug 17 13:03:02.981207939 (211h36m43.712475569s) [upload=220158]: Uploading segments... Aug 17 13:03:02.981400360 (211h36m43.712667981s) [upload=220158]: Got next segment. Inline: false Aug 17 13:03:02.981431599 (211h36m43.712699229s) [upload=220158]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:03:02.981435216 (211h36m43.712702846s) [upload=220158]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:03:03.007542947 (211h36m43.738810577s) [seg_pos={0 0}, upload=220158]: Begin upload segment... Aug 17 13:03:03.007591037 (211h36m43.738858667s) [seg_pos={0 0}, upload=220158]: Done begin upload segment. Aug 17 13:03:03.007729719 (211h36m43.738997338s) [noise=true , address=77.90.31.21:4895 , piece_id=N4QIEGKE2POWLSB2DOEXJU3NN5SVBCBTLFZ67WUXF4KHYBXT3M6A, seg_pos={0 0}, upload=220158]: Uploading piece... Aug 17 13:03:03.013712016 (211h36m43.744979646s) [noise=true , address=77.90.31.10:4524 , piece_id=4PFMJQ4WLI7FZYLFUEPJPMT4DF644M7JQHBE5OLVJOEPEM4VNJBA, seg_pos={0 0}, upload=220158]: Uploading piece... Aug 17 13:03:03.014166592 (211h36m43.745434222s) [noise=true , address=77.90.31.18:4813 , piece_id=UZMR7XBH6BZJ2HVEQYBBOMZKS3WTB7NG34LB2PMAW4PVNS5G7LQQ, seg_pos={0 0}, upload=220158]: Uploading piece... Aug 17 13:03:03.014523382 (211h36m43.745791012s) [noise=true , address=104.36.82.59:4319 , piece_id=B3VZL3ZZGRIU4PUBLHFHEWKSJNPKSMIMXSVLO3LTQ3HIOR73W4XQ, seg_pos={0 0}, upload=220158]: Uploading piece... Aug 17 13:03:03.014885533 (211h36m43.746153153s) [noise=true , address=66.23.193.175:4556 , piece_id=S23EDDJH7BMJWESOCXMD46TJJKW2OLXVDWYXUVSDSDYVQ32AQHCQ, seg_pos={0 0}, upload=220158]: Uploading piece... Aug 17 13:03:03.015150220 (211h36m43.74641785s) [noise=true , address=104.36.82.11:4122 , piece_id=IELHN5BAF45XEPN5YJJKAX7S4LLVEKD5PDFLSCHPVZW4K2KR24SA, seg_pos={0 0}, upload=220158]: Uploading piece... Aug 17 13:03:03.015505048 (211h36m43.746772678s) [noise=true , address=104.36.82.23:4457 , piece_id=HERNT4JJYNEPBL3ZKR47HDRM7MHTJVUSZXJF3TIDOVVYNXJVIKBQ, seg_pos={0 0}, upload=220158]: Uploading piece... Aug 17 13:03:03.016669859 (211h36m43.747937499s) [noise=true , address=104.36.82.8:4004 , piece_id=LR7VRFRXWBW4EVLNXOYN5C5WZ3FWDVQW7KN6GJOE36WKX4YAVRGA, seg_pos={0 0}, upload=220158]: Uploading piece... Aug 17 13:03:03.017164087 (211h36m43.748431717s) [noise=true , address=104.36.82.49:4010 , piece_id=WX5MO4RUDB36HPD3OMAEZDUYYW64BBN3IE33S6V6PQBXXNAETJWA, seg_pos={0 0}, upload=220158]: Uploading piece... Aug 17 13:03:03.017450125 (211h36m43.748717755s) [noise=true , address=104.36.82.44:4900 , piece_id=EQABVSEH37GBAACBOEGLIX754XLKA3R2GG65PJQ6GXPN4X47UGHQ, seg_pos={0 0}, upload=220158]: Uploading piece... Aug 17 13:03:03.017766821 (211h36m43.749034451s) [noise=true , address=104.36.82.5:4868 , piece_id=YNWMCSLBNE4KAGID5CRGKGNRHVVUDDMMDYSM7VJBPXCOJCTISKUA, seg_pos={0 0}, upload=220158]: Uploading piece... Aug 17 13:03:03.022675792 (211h36m43.753943422s) [noise=true , address=77.90.31.16:4752 , piece_id=HB5JMWA7CGOTB2CHV7L7VUWI4LJ4HL7ED2SV77DRUGKOVQPB4IUQ, seg_pos={0 0}, upload=220158]: Uploading piece... Aug 17 13:03:03.025365659 (211h36m43.75663329s) [noise=true , address=156.146.43.225:4456 , piece_id=M33SAA3545FDW3QG7URQQ7NUSX2JS6QD5I46SQMGV542IH2MHBEQ, seg_pos={0 0}, upload=220158]: Uploading piece... Aug 17 13:03:03.025851022 (211h36m43.757118672s) [noise=true , address=104.36.82.2:4740 , piece_id=ROA4R2BY5NHTDL4OBRYZT34T22AWJMLD64BQSM4EVDS76EIS3WQA, seg_pos={0 0}, upload=220158]: Uploading piece... Aug 17 13:03:03.026459245 (211h36m43.757726866s) [noise=true , address=77.90.31.19:4869 , piece_id=X7BX2U5XRPFNJ6SOHQ467ZORNJBWGFZWED2BBVQNXQIJLHMUXHQA, seg_pos={0 0}, upload=220158]: Uploading piece... Aug 17 13:03:03.026978411 (211h36m43.758246041s) [noise=true , address=45.91.116.8:4062 , piece_id=PBLSUI3HS4QLPJDX7LNZQAO3DIHOHKPCESIZ36A7NOSI4DRJTKBQ, seg_pos={0 0}, upload=220158]: Uploading piece... Aug 17 13:03:03.027564193 (211h36m43.758831823s) [noise=true , address=79.127.201.215:4345 , piece_id=X22UESFN7VUUAN5GBVPGMTAZBALDIRKSJR5ECDE2D5YH4DQJXIJA, seg_pos={0 0}, upload=220158]: Uploading piece... Aug 17 13:03:03.028148171 (211h36m43.759415801s) [noise=true , address=104.36.82.58:4298 , piece_id=HL3KNRYNGHONNYXZV3PM5XMVXK5UWW4TZ6R75Z34AOU35BRKFXUA, seg_pos={0 0}, upload=220158]: Uploading piece... Aug 17 13:03:03.108544161 (211h36m43.839811781s) [upload=220158]: Next returned nil segment Aug 17 13:03:03.108558949 (211h36m43.839826568s) [upload=220158]: Waiting for error group managing segments... Aug 17 13:03:03.150775709 (211h36m43.882043329s) [noise=true , address=77.90.31.19:4869 , piece_id=X7BX2U5XRPFNJ6SOHQ467ZORNJBWGFZWED2BBVQNXQIJLHMUXHQA, seg_pos={0 0}, upload=220158]: Done uploading piece. err: <nil> Aug 17 13:03:03.179577296 (211h36m43.910844926s) [noise=true , address=45.91.116.8:4062 , piece_id=PBLSUI3HS4QLPJDX7LNZQAO3DIHOHKPCESIZ36A7NOSI4DRJTKBQ, seg_pos={0 0}, upload=220158]: Done uploading piece. err: <nil> Aug 17 13:03:03.184956360 (211h36m43.91622398s) [noise=true , address=77.90.31.21:4895 , piece_id=N4QIEGKE2POWLSB2DOEXJU3NN5SVBCBTLFZ67WUXF4KHYBXT3M6A, seg_pos={0 0}, upload=220158]: Done uploading piece. err: <nil> Aug 17 13:03:03.229929925 (211h36m43.961197555s) [noise=true , address=77.90.31.10:4524 , piece_id=4PFMJQ4WLI7FZYLFUEPJPMT4DF644M7JQHBE5OLVJOEPEM4VNJBA, seg_pos={0 0}, upload=220158]: Done uploading piece. err: <nil> Aug 17 13:03:03.272370286 (211h36m44.003637916s) [noise=true , address=79.127.201.215:4345 , piece_id=X22UESFN7VUUAN5GBVPGMTAZBALDIRKSJR5ECDE2D5YH4DQJXIJA, seg_pos={0 0}, upload=220158]: Done uploading piece. err: <nil> Aug 17 13:03:03.278402578 (211h36m44.009670199s) [noise=true , address=77.90.31.16:4752 , piece_id=HB5JMWA7CGOTB2CHV7L7VUWI4LJ4HL7ED2SV77DRUGKOVQPB4IUQ, seg_pos={0 0}, upload=220158]: Done uploading piece. err: <nil> Aug 17 13:03:03.289564617 (211h36m44.020832237s) [noise=true , address=156.146.43.225:4456 , piece_id=M33SAA3545FDW3QG7URQQ7NUSX2JS6QD5I46SQMGV542IH2MHBEQ, seg_pos={0 0}, upload=220158]: Done uploading piece. err: <nil> Aug 17 13:03:03.291202778 (211h36m44.022470408s) [noise=true , address=77.90.31.18:4813 , piece_id=UZMR7XBH6BZJ2HVEQYBBOMZKS3WTB7NG34LB2PMAW4PVNS5G7LQQ, seg_pos={0 0}, upload=220158]: Done uploading piece. err: <nil> Aug 17 13:03:03.627093107 (211h36m44.358360737s) [noise=true , address=66.23.193.175:4556 , piece_id=S23EDDJH7BMJWESOCXMD46TJJKW2OLXVDWYXUVSDSDYVQ32AQHCQ, seg_pos={0 0}, upload=220158]: Done uploading piece. err: <nil> Aug 17 13:04:02.964151204 (211h37m43.695418864s) [noise=true , address=104.36.82.5:4868 , piece_id=YNWMCSLBNE4KAGID5CRGKGNRHVVUDDMMDYSM7VJBPXCOJCTISKUA, seg_pos={0 0}, upload=220158]: Done uploading piece. err: ecclient: upload failed (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v, address:104.36.82.5:4868): protocol: rpc: dial tcp 104.36.82.5:4868: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:04:02.964265369 (211h37m43.695533019s) [noise=true , address=104.36.82.23:4457 , piece_id=HERNT4JJYNEPBL3ZKR47HDRM7MHTJVUSZXJF3TIDOVVYNXJVIKBQ, seg_pos={0 0}, upload=220158]: Done uploading piece. err: ecclient: upload failed (node:12mpuVB9gpUQ6dNLaMAVaDkgctVSetSbPRxhxU49WhAzycwwNhs, address:104.36.82.23:4457): protocol: rpc: dial tcp 104.36.82.23:4457: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:04:02.964321995 (211h37m43.695589655s) [noise=true , address=104.36.82.8:4004 , piece_id=LR7VRFRXWBW4EVLNXOYN5C5WZ3FWDVQW7KN6GJOE36WKX4YAVRGA, seg_pos={0 0}, upload=220158]: Done uploading piece. err: ecclient: upload failed (node:12BdVmmJ1L1sEMcHXTAdXBXU6dmdhUE9VT74jCNdZj5bMqJWxqp, address:104.36.82.8:4004): protocol: rpc: dial tcp 104.36.82.8:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:04:02.964380194 (211h37m43.695647855s) [noise=true , address=104.36.82.59:4319 , piece_id=B3VZL3ZZGRIU4PUBLHFHEWKSJNPKSMIMXSVLO3LTQ3HIOR73W4XQ, seg_pos={0 0}, upload=220158]: Done uploading piece. err: ecclient: upload failed (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK, address:104.36.82.59:4319): protocol: write tcp 5.161.89.252:60488->104.36.82.59:4319: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:04:02.964392948 (211h37m43.695660598s) [noise=true , address=104.36.82.11:4122 , piece_id=IELHN5BAF45XEPN5YJJKAX7S4LLVEKD5PDFLSCHPVZW4K2KR24SA, seg_pos={0 0}, upload=220158]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:04:02.964432944 (211h37m43.695700595s) [noise=true , address=104.36.82.49:4010 , piece_id=WX5MO4RUDB36HPD3OMAEZDUYYW64BBN3IE33S6V6PQBXXNAETJWA, seg_pos={0 0}, upload=220158]: Done uploading piece. err: ecclient: upload failed (node:125Jkbj6bsEyRKAhrZPH1Hso9Eaou6Ly6qHKpKFqvK8giPbknjY, address:104.36.82.49:4010): protocol: write tcp 5.161.89.252:48242->104.36.82.49:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:04:02.964452281 (211h37m43.69571993s) [noise=true , address=104.36.82.2:4740 , piece_id=ROA4R2BY5NHTDL4OBRYZT34T22AWJMLD64BQSM4EVDS76EIS3WQA, seg_pos={0 0}, upload=220158]: Done uploading piece. err: ecclient: upload failed (node:12wgQDbE17gP8c6kbKnNyywSkY8Hk7mchcMaymLBQzPwRsSB8QE, address:104.36.82.2:4740): protocol: rpc: dial tcp 104.36.82.2:4740: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:04:02.964482016 (211h37m43.695749666s) [noise=true , address=104.36.82.44:4900 , piece_id=EQABVSEH37GBAACBOEGLIX754XLKA3R2GG65PJQ6GXPN4X47UGHQ, seg_pos={0 0}, upload=220158]: Done uploading piece. err: ecclient: upload failed (node:1R8K65zsCspu7W2oAnvtpz7zEsQFkDG1e5BAp6faQYMQrkh4oQ, address:104.36.82.44:4900): protocol: rpc: dial tcp 104.36.82.44:4900: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:04:02.964523564 (211h37m43.695791214s) [noise=true , address=104.36.82.58:4298 , piece_id=HL3KNRYNGHONNYXZV3PM5XMVXK5UWW4TZ6R75Z34AOU35BRKFXUA, seg_pos={0 0}, upload=220158]: Done uploading piece. err: ecclient: upload failed (node:1MWsyGhKkDhjngUDDRvxmhQddbNnRZ1A3mpzEtjLs9ZcRC5LBz, address:104.36.82.58:4298): protocol: write tcp 5.161.89.252:17594->104.36.82.58:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:04:02.964573297 (211h37m43.695840948s) [seg_pos={0 0}, upload=220158]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil> Aug 17 13:04:02.979409854 (211h37m43.710677504s) [upload=220158]: Done uploading segments... Aug 17 13:04:02.979426887 (211h37m43.710694537s) [upload=220158]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9) Duration: 1m0.017901809s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........................................X..............X....] duration: 119.740279ms FAIL (2025-08-17 13:03:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 119.764655ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................XX.] duration: 1.588829164s FAIL (2025-08-17 13:03:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 9) ======LOGS====== SUHOJ7LMB7E42LTVIBEI3FGYWQ, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.565350188 (211h36m28.296617838s) [noise=true , address=77.90.31.10:4523 , piece_id=MMK734YEAUPZNPFNCXL6F37GPBLNK2ZJRCPTHXJCOR3C76W3DXRA, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.565472249 (211h36m28.296739908s) [noise=true , address=104.36.82.3:4770 , piece_id=G35BLJFPICUO2TJP5LGNYHFVBYI667PB6GS4UMXGRHIABYGGNDUA, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.565868043 (211h36m28.297135692s) [noise=true , address=77.90.31.21:4901 , piece_id=OWRLTQBYVC743SVJ7EQVKE6QZKFZKEUE4OVCHNK5BJ5NIY42NGUQ, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.566075072 (211h36m28.297342752s) [noise=true , address=104.36.82.60:4350 , piece_id=4YOTOJE5RDUGYY3TOTJX6DDYKVWPOQKV2KCMQSFIXVQ7L6EAKDUQ, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.566346522 (211h36m28.297614182s) [noise=true , address=104.36.82.59:4315 , piece_id=OBSDIGRWGDN6PPE6K424QOKQMWJ2HEITXCPCZT5UOP5NNGYTCSGQ, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.566597504 (211h36m28.297865154s) [noise=true , address=45.91.116.12:4228 , piece_id=KAFIK3N4J2ZVNUPJPFN66V2WMH6EHWDRX3UDXFYAB2WRVJOZ2XGQ, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.566743248 (211h36m28.298010898s) [noise=true , address=77.90.31.11:4565 , piece_id=OQOTLKCAXZFWFQ7V42PNVDSF64GGU7UGPNWDZVT27SYCGBSAVGPQ, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.567108445 (211h36m28.298376095s) [noise=true , address=45.91.116.11:4146 , piece_id=GPKZ4EXE4P6HMVXXMC54M2JFUFIRHMO4AOJILCNYODSACEKZPFDA, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.567218942 (211h36m28.298486592s) [noise=true , address=104.36.82.5:4859 , piece_id=UQA4IE4KHKHS2OGWNA7QZN67WJW7A3OEG7CVVVRORTV64HYGJC3Q, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.567544795 (211h36m28.298812445s) [noise=true , address=104.36.82.11:4133 , piece_id=3JI53CESM5W4TGILBZMLYTGHMBGF7MO4D5SPIA7N6YWS7P5L4LQQ, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.567947433 (211h36m28.299215093s) [noise=true , address=66.23.193.175:4560 , piece_id=XOW7CK5GQU54ZKQCL65YMXI26C2VLLJLURUEUMYJRVYLZ3GNZAGQ, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.568301427 (211h36m28.299569137s) [noise=true , address=45.91.116.7:4042 , piece_id=3Z43HDRLS4J567NKIQAXU52UWAI7KJHLGGFJEPHWA73WYI5BSVAQ, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.568597955 (211h36m28.299865596s) [noise=true , address=104.36.82.21:4499 , piece_id=KPNOAHKMXL4KUPG5E5HELU2TTMX2FJBRCY6KFMR7DFPGBFW7PY6Q, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.568962291 (211h36m28.300229941s) [noise=true , address=45.91.116.4:4969 , piece_id=SDBPJ43HOAK27ZA7HVRF3G747YH7SJMBLPCTLVWB4S2YYU33MXXA, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.569117732 (211h36m28.300385383s) [noise=true , address=104.36.82.23:4453 , piece_id=4YEDTSHE6YPKQEIOAKEXX34RHIKQMDP4T4QMZHKUWGUCVCQTBSWQ, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.569591755 (211h36m28.300859414s) [noise=true , address=77.90.31.19:4859 , piece_id=DVS5CHUBGAKBBXLSXBMY5NY4EZS7Z2UXJQN4FRU2TJL6CHL72XSA, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.650343724 (211h36m28.381611354s) [upload=220156]: Next returned nil segment Aug 17 13:02:47.650355626 (211h36m28.381623246s) [upload=220156]: Waiting for error group managing segments... Aug 17 13:02:47.658712279 (211h36m28.389979909s) [noise=true , address=45.91.116.4:4969 , piece_id=SDBPJ43HOAK27ZA7HVRF3G747YH7SJMBLPCTLVWB4S2YYU33MXXA, seg_pos={0 0}, upload=220156]: Done uploading piece. err: <nil> Aug 17 13:02:47.659997686 (211h36m28.391265346s) [noise=true , address=45.91.116.12:4228 , piece_id=KAFIK3N4J2ZVNUPJPFN66V2WMH6EHWDRX3UDXFYAB2WRVJOZ2XGQ, seg_pos={0 0}, upload=220156]: Done uploading piece. err: <nil> Aug 17 13:02:47.679178863 (211h36m28.410446483s) [noise=true , address=45.91.116.7:4042 , piece_id=3Z43HDRLS4J567NKIQAXU52UWAI7KJHLGGFJEPHWA73WYI5BSVAQ, seg_pos={0 0}, upload=220156]: Done uploading piece. err: <nil> Aug 17 13:02:47.684822355 (211h36m28.416090004s) [noise=true , address=77.90.31.11:4565 , piece_id=OQOTLKCAXZFWFQ7V42PNVDSF64GGU7UGPNWDZVT27SYCGBSAVGPQ, seg_pos={0 0}, upload=220156]: Done uploading piece. err: <nil> Aug 17 13:02:47.698784367 (211h36m28.430052017s) [noise=true , address=77.90.31.21:4901 , piece_id=OWRLTQBYVC743SVJ7EQVKE6QZKFZKEUE4OVCHNK5BJ5NIY42NGUQ, seg_pos={0 0}, upload=220156]: Done uploading piece. err: <nil> Aug 17 13:02:47.727357184 (211h36m28.458624834s) [noise=true , address=77.90.31.10:4523 , piece_id=MMK734YEAUPZNPFNCXL6F37GPBLNK2ZJRCPTHXJCOR3C76W3DXRA, seg_pos={0 0}, upload=220156]: Done uploading piece. err: <nil> Aug 17 13:02:47.733814365 (211h36m28.465082015s) [noise=true , address=77.90.31.19:4859 , piece_id=DVS5CHUBGAKBBXLSXBMY5NY4EZS7Z2UXJQN4FRU2TJL6CHL72XSA, seg_pos={0 0}, upload=220156]: Done uploading piece. err: <nil> Aug 17 13:02:47.745083805 (211h36m28.476351455s) [noise=true , address=45.91.116.11:4146 , piece_id=GPKZ4EXE4P6HMVXXMC54M2JFUFIRHMO4AOJILCNYODSACEKZPFDA, seg_pos={0 0}, upload=220156]: Done uploading piece. err: <nil> Aug 17 13:02:48.167593258 (211h36m28.898860878s) [noise=true , address=66.23.193.175:4560 , piece_id=XOW7CK5GQU54ZKQCL65YMXI26C2VLLJLURUEUMYJRVYLZ3GNZAGQ, seg_pos={0 0}, upload=220156]: Done uploading piece. err: <nil> Aug 17 13:03:47.524323914 (211h37m28.255591544s) [noise=true , address=104.36.82.60:4350 , piece_id=4YOTOJE5RDUGYY3TOTJX6DDYKVWPOQKV2KCMQSFIXVQ7L6EAKDUQ, seg_pos={0 0}, upload=220156]: Done uploading piece. err: ecclient: upload failed (node:12byqLui89KiVQB8FetY1aAZAX3Ziy6orZr4DeSgLFjpFQe2CH, address:104.36.82.60:4350): protocol: rpc: dial tcp 104.36.82.60:4350: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:47.524387313 (211h37m28.255654943s) [noise=true , address=104.36.82.4:4832 , piece_id=ZHZ3VTVP2GIWWF7MGK2UJK5YPKFCVBIBQCVE4GI4VOHKHAD5TAIQ, seg_pos={0 0}, upload=220156]: Done uploading piece. err: ecclient: upload failed (node:1SWdNMwLYv7KiBqr5Pfo8qvxHnxirHGUDbhZWeojGXwueQUqr9, address:104.36.82.4:4832): protocol: rpc: dial tcp 104.36.82.4:4832: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:47.524435625 (211h37m28.255703255s) [noise=true , address=104.36.82.11:4133 , piece_id=3JI53CESM5W4TGILBZMLYTGHMBGF7MO4D5SPIA7N6YWS7P5L4LQQ, seg_pos={0 0}, upload=220156]: Done uploading piece. err: ecclient: upload failed (node:12G5pngZWYQ2abKHEpM7Pa6YeCQ1LotX4qswbqHUwAy9H7a9Ym2, address:104.36.82.11:4133): protocol: rpc: dial tcp 104.36.82.11:4133: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:47.524467625 (211h37m28.255735255s) [noise=true , address=104.36.82.59:4315 , piece_id=OBSDIGRWGDN6PPE6K424QOKQMWJ2HEITXCPCZT5UOP5NNGYTCSGQ, seg_pos={0 0}, upload=220156]: Done uploading piece. err: ecclient: upload failed (node:1o5mMqSq2uht75TtSp9i4NpnkpA3AbwmNG9ffyp5zZehUqcMuz, address:104.36.82.59:4315): protocol: rpc: dial tcp 104.36.82.59:4315: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:47.524512629 (211h37m28.255780259s) [noise=true , address=104.36.82.44:4896 , piece_id=Z25F6KOTHVFMSUQTBIAUSYVJTVSUHOJ7LMB7E42LTVIBEI3FGYWQ, seg_pos={0 0}, upload=220156]: Done uploading piece. err: ecclient: upload failed (node:12ndSgjgfZMVQni5tw1ugdTrhJt4c4NagdvgYTbSbjatYYtfu1o, address:104.36.82.44:4896): protocol: rpc: dial tcp 104.36.82.44:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:47.524557964 (211h37m28.255825584s) [noise=true , address=104.36.82.3:4770 , piece_id=G35BLJFPICUO2TJP5LGNYHFVBYI667PB6GS4UMXGRHIABYGGNDUA, seg_pos={0 0}, upload=220156]: Done uploading piece. err: ecclient: upload failed (node:1Ug6GpYToYDXiHmiJP4GoE7eJV3kqRveix2S5SLDcKQJtEbNPz, address:104.36.82.3:4770): protocol: rpc: dial tcp 104.36.82.3:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:47.524567593 (211h37m28.255835223s) [noise=true , address=104.36.82.5:4859 , piece_id=UQA4IE4KHKHS2OGWNA7QZN67WJW7A3OEG7CVVVRORTV64HYGJC3Q, seg_pos={0 0}, upload=220156]: Done uploading piece. err: ecclient: upload failed (node:1EiYA6gHiLyHsMfPevj2gLqmqttu1knLw3AM8SbWWtgKEf7Csu, address:104.36.82.5:4859): protocol: rpc: dial tcp 104.36.82.5:4859: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:47.524607448 (211h37m28.255875068s) [noise=true , address=104.36.82.21:4499 , piece_id=KPNOAHKMXL4KUPG5E5HELU2TTMX2FJBRCY6KFMR7DFPGBFW7PY6Q, seg_pos={0 0}, upload=220156]: Done uploading piece. err: ecclient: upload failed (node:1218y8DtJNTTu94dbqGhywmHR4s99zcz9issmMmpgwusr2KgZxc, address:104.36.82.21:4499): protocol: rpc: dial tcp 104.36.82.21:4499: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:47.524626814 (211h37m28.255894444s) [noise=true , address=104.36.82.23:4453 , piece_id=4YEDTSHE6YPKQEIOAKEXX34RHIKQMDP4T4QMZHKUWGUCVCQTBSWQ, seg_pos={0 0}, upload=220156]: Done uploading piece. err: ecclient: upload failed (node:1ktniPbRibXzsTSNW9MbWWxymt9gZKU8sDgTjixc6nYzT9c5tJ, address:104.36.82.23:4453): protocol: write tcp 5.161.89.252:24280->104.36.82.23:4453: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:47.524652473 (211h37m28.255920103s) [seg_pos={0 0}, upload=220156]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil> Aug 17 13:03:47.542610348 (211h37m28.273877978s) [upload=220156]: Done uploading segments... Aug 17 13:03:47.542623353 (211h37m28.273890973s) [upload=220156]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9) Duration: 1.58884839s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 1m0.020026895s FAIL (2025-08-17 13:03:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 9) ======LOGS====== SUHOJ7LMB7E42LTVIBEI3FGYWQ, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.565350188 (211h36m28.296617838s) [noise=true , address=77.90.31.10:4523 , piece_id=MMK734YEAUPZNPFNCXL6F37GPBLNK2ZJRCPTHXJCOR3C76W3DXRA, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.565472249 (211h36m28.296739908s) [noise=true , address=104.36.82.3:4770 , piece_id=G35BLJFPICUO2TJP5LGNYHFVBYI667PB6GS4UMXGRHIABYGGNDUA, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.565868043 (211h36m28.297135692s) [noise=true , address=77.90.31.21:4901 , piece_id=OWRLTQBYVC743SVJ7EQVKE6QZKFZKEUE4OVCHNK5BJ5NIY42NGUQ, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.566075072 (211h36m28.297342752s) [noise=true , address=104.36.82.60:4350 , piece_id=4YOTOJE5RDUGYY3TOTJX6DDYKVWPOQKV2KCMQSFIXVQ7L6EAKDUQ, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.566346522 (211h36m28.297614182s) [noise=true , address=104.36.82.59:4315 , piece_id=OBSDIGRWGDN6PPE6K424QOKQMWJ2HEITXCPCZT5UOP5NNGYTCSGQ, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.566597504 (211h36m28.297865154s) [noise=true , address=45.91.116.12:4228 , piece_id=KAFIK3N4J2ZVNUPJPFN66V2WMH6EHWDRX3UDXFYAB2WRVJOZ2XGQ, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.566743248 (211h36m28.298010898s) [noise=true , address=77.90.31.11:4565 , piece_id=OQOTLKCAXZFWFQ7V42PNVDSF64GGU7UGPNWDZVT27SYCGBSAVGPQ, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.567108445 (211h36m28.298376095s) [noise=true , address=45.91.116.11:4146 , piece_id=GPKZ4EXE4P6HMVXXMC54M2JFUFIRHMO4AOJILCNYODSACEKZPFDA, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.567218942 (211h36m28.298486592s) [noise=true , address=104.36.82.5:4859 , piece_id=UQA4IE4KHKHS2OGWNA7QZN67WJW7A3OEG7CVVVRORTV64HYGJC3Q, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.567544795 (211h36m28.298812445s) [noise=true , address=104.36.82.11:4133 , piece_id=3JI53CESM5W4TGILBZMLYTGHMBGF7MO4D5SPIA7N6YWS7P5L4LQQ, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.567947433 (211h36m28.299215093s) [noise=true , address=66.23.193.175:4560 , piece_id=XOW7CK5GQU54ZKQCL65YMXI26C2VLLJLURUEUMYJRVYLZ3GNZAGQ, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.568301427 (211h36m28.299569137s) [noise=true , address=45.91.116.7:4042 , piece_id=3Z43HDRLS4J567NKIQAXU52UWAI7KJHLGGFJEPHWA73WYI5BSVAQ, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.568597955 (211h36m28.299865596s) [noise=true , address=104.36.82.21:4499 , piece_id=KPNOAHKMXL4KUPG5E5HELU2TTMX2FJBRCY6KFMR7DFPGBFW7PY6Q, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.568962291 (211h36m28.300229941s) [noise=true , address=45.91.116.4:4969 , piece_id=SDBPJ43HOAK27ZA7HVRF3G747YH7SJMBLPCTLVWB4S2YYU33MXXA, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.569117732 (211h36m28.300385383s) [noise=true , address=104.36.82.23:4453 , piece_id=4YEDTSHE6YPKQEIOAKEXX34RHIKQMDP4T4QMZHKUWGUCVCQTBSWQ, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.569591755 (211h36m28.300859414s) [noise=true , address=77.90.31.19:4859 , piece_id=DVS5CHUBGAKBBXLSXBMY5NY4EZS7Z2UXJQN4FRU2TJL6CHL72XSA, seg_pos={0 0}, upload=220156]: Uploading piece... Aug 17 13:02:47.650343724 (211h36m28.381611354s) [upload=220156]: Next returned nil segment Aug 17 13:02:47.650355626 (211h36m28.381623246s) [upload=220156]: Waiting for error group managing segments... Aug 17 13:02:47.658712279 (211h36m28.389979909s) [noise=true , address=45.91.116.4:4969 , piece_id=SDBPJ43HOAK27ZA7HVRF3G747YH7SJMBLPCTLVWB4S2YYU33MXXA, seg_pos={0 0}, upload=220156]: Done uploading piece. err: <nil> Aug 17 13:02:47.659997686 (211h36m28.391265346s) [noise=true , address=45.91.116.12:4228 , piece_id=KAFIK3N4J2ZVNUPJPFN66V2WMH6EHWDRX3UDXFYAB2WRVJOZ2XGQ, seg_pos={0 0}, upload=220156]: Done uploading piece. err: <nil> Aug 17 13:02:47.679178863 (211h36m28.410446483s) [noise=true , address=45.91.116.7:4042 , piece_id=3Z43HDRLS4J567NKIQAXU52UWAI7KJHLGGFJEPHWA73WYI5BSVAQ, seg_pos={0 0}, upload=220156]: Done uploading piece. err: <nil> Aug 17 13:02:47.684822355 (211h36m28.416090004s) [noise=true , address=77.90.31.11:4565 , piece_id=OQOTLKCAXZFWFQ7V42PNVDSF64GGU7UGPNWDZVT27SYCGBSAVGPQ, seg_pos={0 0}, upload=220156]: Done uploading piece. err: <nil> Aug 17 13:02:47.698784367 (211h36m28.430052017s) [noise=true , address=77.90.31.21:4901 , piece_id=OWRLTQBYVC743SVJ7EQVKE6QZKFZKEUE4OVCHNK5BJ5NIY42NGUQ, seg_pos={0 0}, upload=220156]: Done uploading piece. err: <nil> Aug 17 13:02:47.727357184 (211h36m28.458624834s) [noise=true , address=77.90.31.10:4523 , piece_id=MMK734YEAUPZNPFNCXL6F37GPBLNK2ZJRCPTHXJCOR3C76W3DXRA, seg_pos={0 0}, upload=220156]: Done uploading piece. err: <nil> Aug 17 13:02:47.733814365 (211h36m28.465082015s) [noise=true , address=77.90.31.19:4859 , piece_id=DVS5CHUBGAKBBXLSXBMY5NY4EZS7Z2UXJQN4FRU2TJL6CHL72XSA, seg_pos={0 0}, upload=220156]: Done uploading piece. err: <nil> Aug 17 13:02:47.745083805 (211h36m28.476351455s) [noise=true , address=45.91.116.11:4146 , piece_id=GPKZ4EXE4P6HMVXXMC54M2JFUFIRHMO4AOJILCNYODSACEKZPFDA, seg_pos={0 0}, upload=220156]: Done uploading piece. err: <nil> Aug 17 13:02:48.167593258 (211h36m28.898860878s) [noise=true , address=66.23.193.175:4560 , piece_id=XOW7CK5GQU54ZKQCL65YMXI26C2VLLJLURUEUMYJRVYLZ3GNZAGQ, seg_pos={0 0}, upload=220156]: Done uploading piece. err: <nil> Aug 17 13:03:47.524323914 (211h37m28.255591544s) [noise=true , address=104.36.82.60:4350 , piece_id=4YOTOJE5RDUGYY3TOTJX6DDYKVWPOQKV2KCMQSFIXVQ7L6EAKDUQ, seg_pos={0 0}, upload=220156]: Done uploading piece. err: ecclient: upload failed (node:12byqLui89KiVQB8FetY1aAZAX3Ziy6orZr4DeSgLFjpFQe2CH, address:104.36.82.60:4350): protocol: rpc: dial tcp 104.36.82.60:4350: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:47.524387313 (211h37m28.255654943s) [noise=true , address=104.36.82.4:4832 , piece_id=ZHZ3VTVP2GIWWF7MGK2UJK5YPKFCVBIBQCVE4GI4VOHKHAD5TAIQ, seg_pos={0 0}, upload=220156]: Done uploading piece. err: ecclient: upload failed (node:1SWdNMwLYv7KiBqr5Pfo8qvxHnxirHGUDbhZWeojGXwueQUqr9, address:104.36.82.4:4832): protocol: rpc: dial tcp 104.36.82.4:4832: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:47.524435625 (211h37m28.255703255s) [noise=true , address=104.36.82.11:4133 , piece_id=3JI53CESM5W4TGILBZMLYTGHMBGF7MO4D5SPIA7N6YWS7P5L4LQQ, seg_pos={0 0}, upload=220156]: Done uploading piece. err: ecclient: upload failed (node:12G5pngZWYQ2abKHEpM7Pa6YeCQ1LotX4qswbqHUwAy9H7a9Ym2, address:104.36.82.11:4133): protocol: rpc: dial tcp 104.36.82.11:4133: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:47.524467625 (211h37m28.255735255s) [noise=true , address=104.36.82.59:4315 , piece_id=OBSDIGRWGDN6PPE6K424QOKQMWJ2HEITXCPCZT5UOP5NNGYTCSGQ, seg_pos={0 0}, upload=220156]: Done uploading piece. err: ecclient: upload failed (node:1o5mMqSq2uht75TtSp9i4NpnkpA3AbwmNG9ffyp5zZehUqcMuz, address:104.36.82.59:4315): protocol: rpc: dial tcp 104.36.82.59:4315: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:47.524512629 (211h37m28.255780259s) [noise=true , address=104.36.82.44:4896 , piece_id=Z25F6KOTHVFMSUQTBIAUSYVJTVSUHOJ7LMB7E42LTVIBEI3FGYWQ, seg_pos={0 0}, upload=220156]: Done uploading piece. err: ecclient: upload failed (node:12ndSgjgfZMVQni5tw1ugdTrhJt4c4NagdvgYTbSbjatYYtfu1o, address:104.36.82.44:4896): protocol: rpc: dial tcp 104.36.82.44:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:47.524557964 (211h37m28.255825584s) [noise=true , address=104.36.82.3:4770 , piece_id=G35BLJFPICUO2TJP5LGNYHFVBYI667PB6GS4UMXGRHIABYGGNDUA, seg_pos={0 0}, upload=220156]: Done uploading piece. err: ecclient: upload failed (node:1Ug6GpYToYDXiHmiJP4GoE7eJV3kqRveix2S5SLDcKQJtEbNPz, address:104.36.82.3:4770): protocol: rpc: dial tcp 104.36.82.3:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:47.524567593 (211h37m28.255835223s) [noise=true , address=104.36.82.5:4859 , piece_id=UQA4IE4KHKHS2OGWNA7QZN67WJW7A3OEG7CVVVRORTV64HYGJC3Q, seg_pos={0 0}, upload=220156]: Done uploading piece. err: ecclient: upload failed (node:1EiYA6gHiLyHsMfPevj2gLqmqttu1knLw3AM8SbWWtgKEf7Csu, address:104.36.82.5:4859): protocol: rpc: dial tcp 104.36.82.5:4859: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:47.524607448 (211h37m28.255875068s) [noise=true , address=104.36.82.21:4499 , piece_id=KPNOAHKMXL4KUPG5E5HELU2TTMX2FJBRCY6KFMR7DFPGBFW7PY6Q, seg_pos={0 0}, upload=220156]: Done uploading piece. err: ecclient: upload failed (node:1218y8DtJNTTu94dbqGhywmHR4s99zcz9issmMmpgwusr2KgZxc, address:104.36.82.21:4499): protocol: rpc: dial tcp 104.36.82.21:4499: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:47.524626814 (211h37m28.255894444s) [noise=true , address=104.36.82.23:4453 , piece_id=4YEDTSHE6YPKQEIOAKEXX34RHIKQMDP4T4QMZHKUWGUCVCQTBSWQ, seg_pos={0 0}, upload=220156]: Done uploading piece. err: ecclient: upload failed (node:1ktniPbRibXzsTSNW9MbWWxymt9gZKU8sDgTjixc6nYzT9c5tJ, address:104.36.82.23:4453): protocol: write tcp 5.161.89.252:24280->104.36.82.23:4453: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:47.524652473 (211h37m28.255920103s) [seg_pos={0 0}, upload=220156]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil> Aug 17 13:03:47.542610348 (211h37m28.273877978s) [upload=220156]: Done uploading segments... Aug 17 13:03:47.542623353 (211h37m28.273890973s) [upload=220156]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9) Duration: 1m0.020051821s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 30.026514128s FAIL (2025-08-17 13:03:42 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== 2 (211h36m53.509171912s) [upload=220163]: Got next segment. Inline: false Aug 17 13:03:12.777932475 (211h36m53.509200105s) [upload=220163]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:03:12.777937705 (211h36m53.509205335s) [upload=220163]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:03:12.808643271 (211h36m53.539910921s) [seg_pos={0 0}, upload=220163]: Begin upload segment... Aug 17 13:03:12.808684167 (211h36m53.539951818s) [seg_pos={0 0}, upload=220163]: Done begin upload segment. Aug 17 13:03:12.808715417 (211h36m53.539983067s) [noise=true , address=45.91.116.12:4227 , piece_id=NM3YLA7BCXLTR7WPYOIWGSPEEUUEP5FW4O4MQ2KMHDH2TMK55PKQ, seg_pos={0 0}, upload=220163]: Uploading piece... Aug 17 13:03:12.808728211 (211h36m53.539995861s) [noise=true , address=77.90.31.11:4553 , piece_id=PVLC6Y2LVXTRIBPVPAG7FCWVLMSMTB4GFXPTHSJ5GOBCLIDVXZVA, seg_pos={0 0}, upload=220163]: Uploading piece... Aug 17 13:03:12.808771953 (211h36m53.540039603s) [noise=true , address=45.91.116.11:4117 , piece_id=JPLXJUVOZ7ARS7MJW3SX4CYVGA3NU2JL65UOLDGEMY42UCHDGEOA, seg_pos={0 0}, upload=220163]: Uploading piece... Aug 17 13:03:12.809170833 (211h36m53.540438483s) [noise=true , address=104.36.82.3:4767 , piece_id=MBS3QWYYWIKDWNBAOWY5FHCUHPPUQULZFLO4HAAENQWIR2PAMILA, seg_pos={0 0}, upload=220163]: Uploading piece... Aug 17 13:03:12.809291239 (211h36m53.540558889s) [noise=true , address=104.36.82.9:4005 , piece_id=JN5B4HOZ4UHX56ZZ5VHJUU5VUAJSEMVALRP4BLKLKAKJY4JGRLUA, seg_pos={0 0}, upload=220163]: Uploading piece... Aug 17 13:03:12.809567489 (211h36m53.540835149s) [noise=true , address=104.36.82.12:4151 , piece_id=USJT4VX3COZVYU3NTY56UKZ3EVC46SQNGZH4KZWPDWLFRJ3W2KUA, seg_pos={0 0}, upload=220163]: Uploading piece... Aug 17 13:03:12.809723762 (211h36m53.540991413s) [noise=true , address=104.36.82.5:4839 , piece_id=R26N76PT7EKH7WMVUVHRSZK4EHD6AW3CKB7WFMSPV2776TFMPWFA, seg_pos={0 0}, upload=220163]: Uploading piece... Aug 17 13:03:12.809923358 (211h36m53.541191007s) [noise=true , address=104.36.82.2:4729 , piece_id=67JANHR45BN2IUFL4BIWIUEMX2A6ZGBVRMSCIJR2XZ5KJWGNKN5Q, seg_pos={0 0}, upload=220163]: Uploading piece... Aug 17 13:03:12.810305496 (211h36m53.541573147s) [noise=true , address=45.91.116.10:4082 , piece_id=74DPXCQQXFO4OMYBVTK3GXAOJEU4BNAO42WN5FCWRUMUUIU2PYRA, seg_pos={0 0}, upload=220163]: Uploading piece... Aug 17 13:03:12.810683287 (211h36m53.541950947s) [noise=true , address=104.36.82.12:4150 , piece_id=RAAEW46NVIOWYA3H34OSCRBEJNN53UDAASXKXW7L3W3AANXIM24A, seg_pos={0 0}, upload=220163]: Uploading piece... Aug 17 13:03:12.811045278 (211h36m53.542312927s) [noise=true , address=45.91.116.14:4189 , piece_id=ZXG644JDCKFZN3DZ26VOFQYGDR2777POWEKO4YM6TQIY2I4ZESUQ, seg_pos={0 0}, upload=220163]: Uploading piece... Aug 17 13:03:12.811427456 (211h36m53.542695107s) [noise=true , address=45.91.116.4:4940 , piece_id=YMHB4E7XHAAXM2JWKPZPKB6VBPDPC4PUDR2ZZ5GKCOBCS45MSDOQ, seg_pos={0 0}, upload=220163]: Uploading piece... Aug 17 13:03:12.811793204 (211h36m53.543060854s) [noise=true , address=104.36.82.8:4971 , piece_id=KISROTKM2UY4KZ2CSGPEK52EEBCTDSN7ZWNFSMEFTN2LAPVTU2KQ, seg_pos={0 0}, upload=220163]: Uploading piece... Aug 17 13:03:12.812384716 (211h36m53.543652376s) [noise=true , address=104.36.82.14:4223 , piece_id=6VYG6WAMBFEI2QKP5U46VQPQTWNCYLHCO4JQBQCDTUHMW5UYC4YA, seg_pos={0 0}, upload=220163]: Uploading piece... Aug 17 13:03:12.812789006 (211h36m53.544056666s) [noise=true , address=45.91.116.13:4154 , piece_id=KIIMTMPC444ZRFXPGHXCGTE43E47TLQRUSCWOVWF2TQP37CG3QLA, seg_pos={0 0}, upload=220163]: Uploading piece... Aug 17 13:03:12.813282184 (211h36m53.544549834s) [noise=true , address=104.36.82.10:4077 , piece_id=FANAMNYS5JN6UBLOTMGI4CEVL3EHLM3ZW3DQFEQZYJCQ3N7TYVXA, seg_pos={0 0}, upload=220163]: Uploading piece... Aug 17 13:03:12.914451123 (211h36m53.645718774s) [upload=220163]: Next returned nil segment Aug 17 13:03:12.914466722 (211h36m53.645734373s) [upload=220163]: Waiting for error group managing segments... Aug 17 13:03:12.922835217 (211h36m53.654102878s) [noise=true , address=45.91.116.10:4082 , piece_id=74DPXCQQXFO4OMYBVTK3GXAOJEU4BNAO42WN5FCWRUMUUIU2PYRA, seg_pos={0 0}, upload=220163]: Done uploading piece. err: <nil> Aug 17 13:03:12.923493186 (211h36m53.654760846s) [noise=true , address=45.91.116.14:4189 , piece_id=ZXG644JDCKFZN3DZ26VOFQYGDR2777POWEKO4YM6TQIY2I4ZESUQ, seg_pos={0 0}, upload=220163]: Done uploading piece. err: <nil> Aug 17 13:03:12.923690426 (211h36m53.654958077s) [noise=true , address=45.91.116.11:4117 , piece_id=JPLXJUVOZ7ARS7MJW3SX4CYVGA3NU2JL65UOLDGEMY42UCHDGEOA, seg_pos={0 0}, upload=220163]: Done uploading piece. err: <nil> Aug 17 13:03:12.924596640 (211h36m53.655864291s) [noise=true , address=45.91.116.4:4940 , piece_id=YMHB4E7XHAAXM2JWKPZPKB6VBPDPC4PUDR2ZZ5GKCOBCS45MSDOQ, seg_pos={0 0}, upload=220163]: Done uploading piece. err: <nil> Aug 17 13:03:12.936199959 (211h36m53.667467608s) [noise=true , address=45.91.116.13:4154 , piece_id=KIIMTMPC444ZRFXPGHXCGTE43E47TLQRUSCWOVWF2TQP37CG3QLA, seg_pos={0 0}, upload=220163]: Done uploading piece. err: <nil> Aug 17 13:03:12.940515593 (211h36m53.671783242s) [noise=true , address=45.91.116.12:4227 , piece_id=NM3YLA7BCXLTR7WPYOIWGSPEEUUEP5FW4O4MQ2KMHDH2TMK55PKQ, seg_pos={0 0}, upload=220163]: Done uploading piece. err: <nil> Aug 17 13:03:12.942297303 (211h36m53.673564933s) [noise=true , address=77.90.31.11:4553 , piece_id=PVLC6Y2LVXTRIBPVPAG7FCWVLMSMTB4GFXPTHSJ5GOBCLIDVXZVA, seg_pos={0 0}, upload=220163]: Done uploading piece. err: <nil> Aug 17 13:03:42.766689287 (211h37m23.497956927s) [noise=true , address=104.36.82.3:4767 , piece_id=MBS3QWYYWIKDWNBAOWY5FHCUHPPUQULZFLO4HAAENQWIR2PAMILA, seg_pos={0 0}, upload=220163]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: rpc: dial tcp 104.36.82.3:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:42.766753178 (211h37m23.498020828s) [noise=true , address=104.36.82.12:4151 , piece_id=USJT4VX3COZVYU3NTY56UKZ3EVC46SQNGZH4KZWPDWLFRJ3W2KUA, seg_pos={0 0}, upload=220163]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: rpc: dial tcp 104.36.82.12:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:42.766808622 (211h37m23.498076272s) [noise=true , address=104.36.82.12:4150 , piece_id=RAAEW46NVIOWYA3H34OSCRBEJNN53UDAASXKXW7L3W3AANXIM24A, seg_pos={0 0}, upload=220163]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: rpc: dial tcp 104.36.82.12:4150: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:42.766862433 (211h37m23.498130083s) [noise=true , address=104.36.82.14:4223 , piece_id=6VYG6WAMBFEI2QKP5U46VQPQTWNCYLHCO4JQBQCDTUHMW5UYC4YA, seg_pos={0 0}, upload=220163]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: rpc: dial tcp 104.36.82.14:4223: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:42.766897219 (211h37m23.498164859s) [noise=true , address=104.36.82.2:4729 , piece_id=67JANHR45BN2IUFL4BIWIUEMX2A6ZGBVRMSCIJR2XZ5KJWGNKN5Q, seg_pos={0 0}, upload=220163]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: rpc: dial tcp 104.36.82.2:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:42.766908771 (211h37m23.498176421s) [noise=true , address=104.36.82.9:4005 , piece_id=JN5B4HOZ4UHX56ZZ5VHJUU5VUAJSEMVALRP4BLKLKAKJY4JGRLUA, seg_pos={0 0}, upload=220163]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:42.766953325 (211h37m23.498220975s) [noise=true , address=104.36.82.5:4839 , piece_id=R26N76PT7EKH7WMVUVHRSZK4EHD6AW3CKB7WFMSPV2776TFMPWFA, seg_pos={0 0}, upload=220163]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:104.36.82.5:4839): protocol: rpc: dial tcp 104.36.82.5:4839: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:42.766985414 (211h37m23.498253065s) [noise=true , address=104.36.82.8:4971 , piece_id=KISROTKM2UY4KZ2CSGPEK52EEBCTDSN7ZWNFSMEFTN2LAPVTU2KQ, seg_pos={0 0}, upload=220163]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 5.161.89.252:62632->104.36.82.8:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:42.767000042 (211h37m23.498267683s) [noise=true , address=104.36.82.10:4077 , piece_id=FANAMNYS5JN6UBLOTMGI4CEVL3EHLM3ZW3DQFEQZYJCQ3N7TYVXA, seg_pos={0 0}, upload=220163]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:104.36.82.10:4077): protocol: rpc: dial tcp 104.36.82.10:4077: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:03:42.767027955 (211h37m23.498295604s) [seg_pos={0 0}, upload=220163]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Aug 17 13:03:42.791396294 (211h37m23.522663944s) [upload=220163]: Done uploading segments... Aug 17 13:03:42.791421602 (211h37m23.522689252s) [upload=220163]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 30.026534747s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X..............X...] duration: 194.746927ms FAIL (2025-08-17 13:03:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 194.774588ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [......X................................................X...X] duration: 1m0.025680528s FAIL (2025-08-17 13:02:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:01:59.123387778 (211h35m39.854655409s) [upload=220141]: Starting upload Aug 17 13:01:59.123458722 (211h35m39.854726352s) [upload=220141]: Done starting upload Aug 17 13:01:59.123557368 (211h35m39.854824988s) [upload=220141]: Uploading segments... Aug 17 13:01:59.123697400 (211h35m39.85496503s) [upload=220141]: Got next segment. Inline: false Aug 17 13:01:59.123782822 (211h35m39.855050452s) [upload=220141]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:01:59.123790015 (211h35m39.855057645s) [upload=220141]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:01:59.311656785 (211h35m40.042924405s) [seg_pos={0 0}, upload=220141]: Begin upload segment... Aug 17 13:01:59.311688215 (211h35m40.042955834s) [seg_pos={0 0}, upload=220141]: Done begin upload segment. Aug 17 13:01:59.311706950 (211h35m40.04297457s) [noise=true , address=104.36.82.8:4996 , piece_id=VZMDFM2PFWLJEOCTPTSY6HVI5NKDJIR4TNCJ46WHP7MJOAFD4A5Q, seg_pos={0 0}, upload=220141]: Uploading piece... Aug 17 13:01:59.311741525 (211h35m40.043009174s) [noise=true , address=77.90.31.13:4631 , piece_id=EWVT63KRLDX3E46JYOYNVFHQDG2PH7ILDQR7P3BMZX2ZZWW3POLA, seg_pos={0 0}, upload=220141]: Uploading piece... Aug 17 13:01:59.312060705 (211h35m40.043328325s) [noise=true , address=77.90.31.18:4824 , piece_id=IFTF4DKDO4MQJCLZASHTWMGLH6PXGA22IDQ4FANXK4CHZAV42D3A, seg_pos={0 0}, upload=220141]: Uploading piece... Aug 17 13:01:59.312311076 (211h35m40.043578716s) [noise=true , address=77.90.31.14:4676 , piece_id=OXX64YPLJRFINFLHV7WHKFKB7NXGEN3X2YAFTQ4OUKKTQLMSMHPQ, seg_pos={0 0}, upload=220141]: Uploading piece... Aug 17 13:01:59.312352664 (211h35m40.043620283s) [noise=true , address=45.91.116.10:4100 , piece_id=N5SO3JVHSJ453UGZWJOXXT7DICO52TVZDSBVGYORMJK244N53ARA, seg_pos={0 0}, upload=220141]: Uploading piece... Aug 17 13:01:59.312704646 (211h35m40.043972266s) [noise=true , address=79.127.201.215:4358 , piece_id=FNXCPBRCAXHRJOXFL5RPAQJDYI5FB5JJR4N3VH4GMSHYAS6IATYA, seg_pos={0 0}, upload=220141]: Uploading piece... Aug 17 13:01:59.312717129 (211h35m40.04398478s) [noise=true , address=66.23.193.175:4558 , piece_id=3BKUGLWXJJXAVCUS4JZF6EPF357BIDU5EL24INBWCGYJ2O5K3IOA, seg_pos={0 0}, upload=220141]: Uploading piece... Aug 17 13:01:59.313014578 (211h35m40.044282208s) [noise=true , address=77.90.31.12:4597 , piece_id=Y6LX6PRNRNWGNWMVPLSQG4HX3J3VDTR6265E3CVZFGT6FAYGL2EA, seg_pos={0 0}, upload=220141]: Uploading piece... Aug 17 13:01:59.313106421 (211h35m40.044374072s) [noise=true , address=104.36.82.3:4790 , piece_id=4LRHWWQIQOXF5HKBCJE52BG7AC6UC3OUCF62L2542QFFQ6JTXQUA, seg_pos={0 0}, upload=220141]: Uploading piece... Aug 17 13:01:59.313317398 (211h35m40.044585038s) [noise=true , address=79.127.219.48:4101 , piece_id=XHIYEFOOGSFJNJODHS2ORLKVAZBYX6PWJMUDCVHCBC3IFB5BSR5A, seg_pos={0 0}, upload=220141]: Uploading piece... Aug 17 13:01:59.313595671 (211h35m40.044863301s) [noise=true , address=104.36.82.9:4025 , piece_id=N5C7IT3SFYLPB32SODQKLROAMFLCQLEREYYD3HAA7SSOJXAFOU5Q, seg_pos={0 0}, upload=220141]: Uploading piece... Aug 17 13:01:59.313701100 (211h35m40.04496875s) [noise=true , address=77.90.31.19:4869 , piece_id=B6G3TFC7G3HVW4OJLCKNSLPHVLOP4QERA4R34WC6W5AB6JHSXGUA, seg_pos={0 0}, upload=220141]: Uploading piece... Aug 17 13:01:59.313857655 (211h35m40.045125285s) [noise=true , address=104.36.82.44:4870 , piece_id=7MUFU3I4XHJBVF6PQXJAFCTFK4QCIHTQIEOXUHJ2ADOPB2HTETVQ, seg_pos={0 0}, upload=220141]: Uploading piece... Aug 17 13:01:59.314058090 (211h35m40.045325741s) [noise=true , address=77.90.31.16:4755 , piece_id=XXJ6SB2KI2ND3PB4QAXR2UBLD4KHJHOUO7I3PDDYPBCH6SKF2KHA, seg_pos={0 0}, upload=220141]: Uploading piece... Aug 17 13:01:59.314169610 (211h35m40.045437231s) [noise=true , address=104.36.82.23:4462 , piece_id=7TIWYVU7QMBQOBO2E3UPAPONFB3UALNC445UQHXAWPEW3MEKHHHQ, seg_pos={0 0}, upload=220141]: Uploading piece... Aug 17 13:01:59.314499851 (211h35m40.045767481s) [noise=true , address=104.36.82.60:4338 , piece_id=2HQC4RCEJS2I263K67XZGYRMOZN322BXAHU7HX3IDDLZ4CPWXA4Q, seg_pos={0 0}, upload=220141]: Uploading piece... Aug 17 13:01:59.315346073 (211h35m40.046613703s) [noise=true , address=104.36.82.5:4857 , piece_id=UBG5YWHYALQPMEN3B3GQURJIBOA3BCDEA26BLK4ITECQMSIQTGGA, seg_pos={0 0}, upload=220141]: Uploading piece... Aug 17 13:01:59.315624015 (211h35m40.046891645s) [noise=true , address=104.36.82.39:4758 , piece_id=SAGA3NCTJJ3AZ4UKZ6P76B22PCBXVOZ4QS7VVUI5SPZDO7F72UPA, seg_pos={0 0}, upload=220141]: Uploading piece... Aug 17 13:01:59.397480742 (211h35m40.128748391s) [upload=220141]: Next returned nil segment Aug 17 13:01:59.397499417 (211h35m40.128767077s) [upload=220141]: Waiting for error group managing segments... Aug 17 13:01:59.408033665 (211h35m40.139301314s) [noise=true , address=79.127.219.48:4101 , piece_id=XHIYEFOOGSFJNJODHS2ORLKVAZBYX6PWJMUDCVHCBC3IFB5BSR5A, seg_pos={0 0}, upload=220141]: Done uploading piece. err: <nil> Aug 17 13:01:59.416139285 (211h35m40.147406906s) [noise=true , address=45.91.116.10:4100 , piece_id=N5SO3JVHSJ453UGZWJOXXT7DICO52TVZDSBVGYORMJK244N53ARA, seg_pos={0 0}, upload=220141]: Done uploading piece. err: <nil> Aug 17 13:01:59.426978807 (211h35m40.158246437s) [noise=true , address=77.90.31.19:4869 , piece_id=B6G3TFC7G3HVW4OJLCKNSLPHVLOP4QERA4R34WC6W5AB6JHSXGUA, seg_pos={0 0}, upload=220141]: Done uploading piece. err: <nil> Aug 17 13:01:59.427677631 (211h35m40.158945291s) [noise=true , address=77.90.31.13:4631 , piece_id=EWVT63KRLDX3E46JYOYNVFHQDG2PH7ILDQR7P3BMZX2ZZWW3POLA, seg_pos={0 0}, upload=220141]: Done uploading piece. err: <nil> Aug 17 13:01:59.444699578 (211h35m40.175967229s) [noise=true , address=77.90.31.12:4597 , piece_id=Y6LX6PRNRNWGNWMVPLSQG4HX3J3VDTR6265E3CVZFGT6FAYGL2EA, seg_pos={0 0}, upload=220141]: Done uploading piece. err: <nil> Aug 17 13:01:59.445449990 (211h35m40.17671762s) [noise=true , address=77.90.31.18:4824 , piece_id=IFTF4DKDO4MQJCLZASHTWMGLH6PXGA22IDQ4FANXK4CHZAV42D3A, seg_pos={0 0}, upload=220141]: Done uploading piece. err: <nil> Aug 17 13:01:59.554652134 (211h35m40.285919785s) [noise=true , address=79.127.201.215:4358 , piece_id=FNXCPBRCAXHRJOXFL5RPAQJDYI5FB5JJR4N3VH4GMSHYAS6IATYA, seg_pos={0 0}, upload=220141]: Done uploading piece. err: <nil> Aug 17 13:01:59.673138748 (211h35m40.404406379s) [noise=true , address=77.90.31.16:4755 , piece_id=XXJ6SB2KI2ND3PB4QAXR2UBLD4KHJHOUO7I3PDDYPBCH6SKF2KHA, seg_pos={0 0}, upload=220141]: Done uploading piece. err: <nil> Aug 17 13:01:59.784822178 (211h35m40.516089828s) [noise=true , address=77.90.31.14:4676 , piece_id=OXX64YPLJRFINFLHV7WHKFKB7NXGEN3X2YAFTQ4OUKKTQLMSMHPQ, seg_pos={0 0}, upload=220141]: Done uploading piece. err: <nil> Aug 17 13:01:59.913247810 (211h35m40.64451546s) [noise=true , address=66.23.193.175:4558 , piece_id=3BKUGLWXJJXAVCUS4JZF6EPF357BIDU5EL24INBWCGYJ2O5K3IOA, seg_pos={0 0}, upload=220141]: Done uploading piece. err: <nil> Aug 17 13:02:59.080802224 (211h36m39.812069845s) [noise=true , address=104.36.82.60:4338 , piece_id=2HQC4RCEJS2I263K67XZGYRMOZN322BXAHU7HX3IDDLZ4CPWXA4Q, seg_pos={0 0}, upload=220141]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:104.36.82.60:4338): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:02:59.080872196 (211h36m39.812139816s) [noise=true , address=104.36.82.3:4790 , piece_id=4LRHWWQIQOXF5HKBCJE52BG7AC6UC3OUCF62L2542QFFQ6JTXQUA, seg_pos={0 0}, upload=220141]: Done uploading piece. err: ecclient: upload failed (node:1AowWwVvxzmnW5qwbqbMBVbvNoigffjAN51LQp8e6XA9CZkeXL, address:104.36.82.3:4790): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:02:59.080953349 (211h36m39.812220969s) [noise=true , address=104.36.82.39:4758 , piece_id=SAGA3NCTJJ3AZ4UKZ6P76B22PCBXVOZ4QS7VVUI5SPZDO7F72UPA, seg_pos={0 0}, upload=220141]: Done uploading piece. err: ecclient: upload failed (node:12XjsLCQGSuQkDmyC4QFENshoLeo4kCZG1YNm34wBvG8UEuxTG7, address:104.36.82.39:4758): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:02:59.080973367 (211h36m39.812240997s) [noise=true , address=104.36.82.9:4025 , piece_id=N5C7IT3SFYLPB32SODQKLROAMFLCQLEREYYD3HAA7SSOJXAFOU5Q, seg_pos={0 0}, upload=220141]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: rpc: dial tcp 104.36.82.9:4025: i/o timeout; context deadline exceeded Aug 17 13:02:59.081001700 (211h36m39.812269319s) [noise=true , address=104.36.82.8:4996 , piece_id=VZMDFM2PFWLJEOCTPTSY6HVI5NKDJIR4TNCJ46WHP7MJOAFD4A5Q, seg_pos={0 0}, upload=220141]: Done uploading piece. err: ecclient: upload failed (node:12r6XcE2fb6YxU6kez4UMPeq82jNPi7YiRNmi33tQcTVYMYrMV4, address:104.36.82.8:4996): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:02:59.081021877 (211h36m39.812289498s) [noise=true , address=104.36.82.5:4857 , piece_id=UBG5YWHYALQPMEN3B3GQURJIBOA3BCDEA26BLK4ITECQMSIQTGGA, seg_pos={0 0}, upload=220141]: Done uploading piece. err: ecclient: upload failed (node:1qjxNyfMA6ZjUurQZty2L8U5SUGo6wcLfCb3KVbmWn3ZaDXzQz, address:104.36.82.5:4857): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:02:59.081031957 (211h36m39.812299576s) [noise=true , address=104.36.82.44:4870 , piece_id=7MUFU3I4XHJBVF6PQXJAFCTFK4QCIHTQIEOXUHJ2ADOPB2HTETVQ, seg_pos={0 0}, upload=220141]: Done uploading piece. err: ecclient: upload failed (node:12sDH7ccMQhjePqpbTkkxMAtumNLxpMHiT56cnfaYrUTFYxuPVt, address:104.36.82.44:4870): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:02:59.081063866 (211h36m39.812331487s) [noise=true , address=104.36.82.23:4462 , piece_id=7TIWYVU7QMBQOBO2E3UPAPONFB3UALNC445UQHXAWPEW3MEKHHHQ, seg_pos={0 0}, upload=220141]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: context deadline exceeded; context deadline exceeded Aug 17 13:02:59.081078123 (211h36m39.812345743s) [seg_pos={0 0}, upload=220141]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:02:59.105409985 (211h36m39.836677605s) [upload=220141]: Done uploading segments... Aug 17 13:02:59.105424463 (211h36m39.836692093s) [upload=220141]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1m0.02569203s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.01798215s FAIL (2025-08-17 13:02:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 9) ======LOGS====== upload=220138]: Uploading piece... Aug 17 13:01:47.482185725 (211h35m28.213453386s) [noise=true , address=104.36.82.59:4322 , piece_id=5UWFUV3RSBNVFMR44UUDY77W75UECG3IYZ5ZLBTLESPVRDQIUXJQ, seg_pos={0 0}, upload=220138]: Uploading piece... Aug 17 13:01:47.482314839 (211h35m28.213582498s) [noise=true , address=79.127.220.101:4074 , piece_id=GU6LTP2SRA6YVYV7NRUHSEQGWJXGUPHTEC5NP7M5JWWBBPVSIP6A, seg_pos={0 0}, upload=220138]: Uploading piece... Aug 17 13:01:47.482639919 (211h35m28.213907569s) [noise=true , address=77.90.31.15:4720 , piece_id=6NIZITDUZZ6YMMG43IOUFHP7VUNY3LERG425BFZJHKUHOAGONNIQ, seg_pos={0 0}, upload=220138]: Uploading piece... Aug 17 13:01:47.482746149 (211h35m28.21401381s) [noise=true , address=104.36.82.5:4843 , piece_id=PNKAED3QVYXSC3B2QAXA64DGWJTX23WJ276MH6GBJTLDLYWJKDLQ, seg_pos={0 0}, upload=220138]: Uploading piece... Aug 17 13:01:47.483077553 (211h35m28.214345203s) [noise=true , address=77.90.31.19:4849 , piece_id=Z2D7H4P6LQ3XZSVFT3E3UMDCKOLCTUMR3KFR73T5GUOTCFUMWSHA, seg_pos={0 0}, upload=220138]: Uploading piece... Aug 17 13:01:47.483126354 (211h35m28.214394005s) [noise=true , address=77.90.31.11:4561 , piece_id=EV2E37Q7IUJ2S7DJPQM2E6LH233W4EXOBS46E5TWS5VMRBO5GSMQ, seg_pos={0 0}, upload=220138]: Uploading piece... Aug 17 13:01:47.483462646 (211h35m28.214730296s) [noise=true , address=79.127.201.214:4320 , piece_id=3KVTGHG2LI5OK5RKX6R6B4NMKMCAII23SNKYQA5IZXFMO2WJSLRA, seg_pos={0 0}, upload=220138]: Uploading piece... Aug 17 13:01:47.483547757 (211h35m28.214815407s) [noise=true , address=104.36.82.21:4499 , piece_id=JF54UVGF34TB7YHQCQLX24OSD657BYOQAPGL27LL5LTKEZLALS6A, seg_pos={0 0}, upload=220138]: Uploading piece... Aug 17 13:01:47.483833113 (211h35m28.215100763s) [noise=true , address=79.127.219.48:4099 , piece_id=C6CWGS6X2X5YCSNDPWNGDNNCX7IFI6BPW2AL4K4WNBAI3VYP7VEQ, seg_pos={0 0}, upload=220138]: Uploading piece... Aug 17 13:01:47.484030154 (211h35m28.215297803s) [noise=true , address=104.36.82.44:4902 , piece_id=S3NSHJ5HMNUDRGP3F7PHDG5JAEW4CGUNTBJ2R5FQ73UWSFAIXB3A, seg_pos={0 0}, upload=220138]: Uploading piece... Aug 17 13:01:47.484306623 (211h35m28.215574273s) [noise=true , address=104.36.82.60:4345 , piece_id=YARAFHRDWLJA7RDMDUZ7CUDJ4FBXVRSX7XNZRYNCZQL5PQSZ77XQ, seg_pos={0 0}, upload=220138]: Uploading piece... Aug 17 13:01:47.484549020 (211h35m28.21581667s) [noise=true , address=104.36.82.34:4549 , piece_id=4XNPDUOWGVO24ZEWVAIHXV5BF4CMHPB6X6FACBKJ74SJUDEARWXQ, seg_pos={0 0}, upload=220138]: Uploading piece... Aug 17 13:01:47.484707016 (211h35m28.215974666s) [noise=true , address=77.90.31.18:4830 , piece_id=BCN64SOB6RTWQLLDKEN7VOY4LEYOD43PWOEK5JPAYEQUPEDYDKZQ, seg_pos={0 0}, upload=220138]: Uploading piece... Aug 17 13:01:47.485059289 (211h35m28.216327129s) [noise=true , address=104.36.82.22:4512 , piece_id=QPNEP7GWFPHGOOGQDMYRCBLCY2O77362U4AHMCBQGITP44GVYKNA, seg_pos={0 0}, upload=220138]: Uploading piece... Aug 17 13:01:47.485138527 (211h35m28.216406178s) [noise=true , address=77.90.31.10:4527 , piece_id=S2GDH7LDBTCXDYPN6QJZTJIHDDCIYNSHP6HPAJEMHS6Z4EXOH56Q, seg_pos={0 0}, upload=220138]: Uploading piece... Aug 17 13:01:47.485571963 (211h35m28.216839643s) [noise=true , address=104.36.82.58:4299 , piece_id=7RT5MLYQWDMKBI6D2RLIWSRSCZAM3VRYNXCHZBGWEY24H4O3RVUA, seg_pos={0 0}, upload=220138]: Uploading piece... Aug 17 13:01:47.592421533 (211h35m28.323689163s) [upload=220138]: Next returned nil segment Aug 17 13:01:47.592433956 (211h35m28.323701576s) [upload=220138]: Waiting for error group managing segments... Aug 17 13:01:47.601323331 (211h35m28.332590981s) [noise=true , address=77.90.31.19:4849 , piece_id=Z2D7H4P6LQ3XZSVFT3E3UMDCKOLCTUMR3KFR73T5GUOTCFUMWSHA, seg_pos={0 0}, upload=220138]: Done uploading piece. err: <nil> Aug 17 13:01:47.602161116 (211h35m28.333428737s) [noise=true , address=77.90.31.10:4527 , piece_id=S2GDH7LDBTCXDYPN6QJZTJIHDDCIYNSHP6HPAJEMHS6Z4EXOH56Q, seg_pos={0 0}, upload=220138]: Done uploading piece. err: <nil> Aug 17 13:01:47.602516234 (211h35m28.333783885s) [noise=true , address=79.127.219.48:4099 , piece_id=C6CWGS6X2X5YCSNDPWNGDNNCX7IFI6BPW2AL4K4WNBAI3VYP7VEQ, seg_pos={0 0}, upload=220138]: Done uploading piece. err: <nil> Aug 17 13:01:47.616190356 (211h35m28.347457976s) [noise=true , address=77.90.31.11:4561 , piece_id=EV2E37Q7IUJ2S7DJPQM2E6LH233W4EXOBS46E5TWS5VMRBO5GSMQ, seg_pos={0 0}, upload=220138]: Done uploading piece. err: <nil> Aug 17 13:01:47.634983513 (211h35m28.366251163s) [noise=true , address=77.90.31.15:4720 , piece_id=6NIZITDUZZ6YMMG43IOUFHP7VUNY3LERG425BFZJHKUHOAGONNIQ, seg_pos={0 0}, upload=220138]: Done uploading piece. err: <nil> Aug 17 13:01:47.645296124 (211h35m28.376563774s) [noise=true , address=77.90.31.18:4830 , piece_id=BCN64SOB6RTWQLLDKEN7VOY4LEYOD43PWOEK5JPAYEQUPEDYDKZQ, seg_pos={0 0}, upload=220138]: Done uploading piece. err: <nil> Aug 17 13:01:47.663624639 (211h35m28.394892288s) [noise=true , address=79.127.220.101:4074 , piece_id=GU6LTP2SRA6YVYV7NRUHSEQGWJXGUPHTEC5NP7M5JWWBBPVSIP6A, seg_pos={0 0}, upload=220138]: Done uploading piece. err: <nil> Aug 17 13:01:47.735072172 (211h35m28.466339813s) [noise=true , address=79.127.201.214:4320 , piece_id=3KVTGHG2LI5OK5RKX6R6B4NMKMCAII23SNKYQA5IZXFMO2WJSLRA, seg_pos={0 0}, upload=220138]: Done uploading piece. err: <nil> Aug 17 13:01:48.082648064 (211h35m28.813915714s) [noise=true , address=66.23.193.175:4557 , piece_id=I44X5OMQOWLTUCAV6UGKUF5CNZ2CVEJ6ZFMXQAGZAWGNDZRLVAFQ, seg_pos={0 0}, upload=220138]: Done uploading piece. err: <nil> Aug 17 13:02:47.428211664 (211h36m28.159479284s) [noise=true , address=104.36.82.59:4322 , piece_id=5UWFUV3RSBNVFMR44UUDY77W75UECG3IYZ5ZLBTLESPVRDQIUXJQ, seg_pos={0 0}, upload=220138]: Done uploading piece. err: ecclient: upload failed (node:12bELQDDH69yamGywE32K8YhpPbMRVorkpm85ZFuAtb2CD58ZPH, address:104.36.82.59:4322): protocol: rpc: dial tcp 104.36.82.59:4322: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:02:47.428490177 (211h36m28.159757807s) [noise=true , address=104.36.82.60:4345 , piece_id=YARAFHRDWLJA7RDMDUZ7CUDJ4FBXVRSX7XNZRYNCZQL5PQSZ77XQ, seg_pos={0 0}, upload=220138]: Done uploading piece. err: ecclient: upload failed (node:1KSTyRJ62wcMHx8K7uwqp5e8L8yhxH7V3YtBHnYu6QJ3K3bXpQ, address:104.36.82.60:4345): protocol: rpc: dial tcp 104.36.82.60:4345: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:02:47.428549269 (211h36m28.159816889s) [noise=true , address=104.36.82.23:4463 , piece_id=XMMVLAO7DL4PGONVX2OBPB45NFBMJBAZSWFP2VRFMCVP6SM7PWKA, seg_pos={0 0}, upload=220138]: Done uploading piece. err: ecclient: upload failed (node:12d3ACC8cWirsfXo2XNPTtr6sULutTpdEpvFDweeiQuPQ3NiZyL, address:104.36.82.23:4463): protocol: rpc: dial tcp 104.36.82.23:4463: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:02:47.428808647 (211h36m28.160076267s) [noise=true , address=104.36.82.5:4843 , piece_id=PNKAED3QVYXSC3B2QAXA64DGWJTX23WJ276MH6GBJTLDLYWJKDLQ, seg_pos={0 0}, upload=220138]: Done uploading piece. err: ecclient: upload failed (node:12HvL2JDP4GCsu3tziVEKRMh14nEzyRMRuWNSmVvs27LzHTbNpS, address:104.36.82.5:4843): protocol: write tcp 5.161.89.252:36048->104.36.82.5:4843: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:02:47.429004835 (211h36m28.160272455s) [noise=true , address=104.36.82.21:4499 , piece_id=JF54UVGF34TB7YHQCQLX24OSD657BYOQAPGL27LL5LTKEZLALS6A, seg_pos={0 0}, upload=220138]: Done uploading piece. err: ecclient: upload failed (node:1218y8DtJNTTu94dbqGhywmHR4s99zcz9issmMmpgwusr2KgZxc, address:104.36.82.21:4499): protocol: rpc: dial tcp 104.36.82.21:4499: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:02:47.429159346 (211h36m28.160426967s) [noise=true , address=104.36.82.44:4902 , piece_id=S3NSHJ5HMNUDRGP3F7PHDG5JAEW4CGUNTBJ2R5FQ73UWSFAIXB3A, seg_pos={0 0}, upload=220138]: Done uploading piece. err: ecclient: upload failed (node:1Be9T5kXUwLU3C6wBeKFV3Q7aZFbZDTNG1oaEft8V6rmySUcEE, address:104.36.82.44:4902): protocol: rpc: dial tcp 104.36.82.44:4902: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:02:47.429208829 (211h36m28.160476449s) [noise=true , address=104.36.82.58:4299 , piece_id=7RT5MLYQWDMKBI6D2RLIWSRSCZAM3VRYNXCHZBGWEY24H4O3RVUA, seg_pos={0 0}, upload=220138]: Done uploading piece. err: ecclient: upload failed (node:1RBzddYrypDRDqsdPAi2w4KE2vTiv3kxUTR98ecfYtVpaqqQTp, address:104.36.82.58:4299): protocol: write tcp 5.161.89.252:62040->104.36.82.58:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:02:47.429215622 (211h36m28.160483242s) [noise=true , address=104.36.82.22:4512 , piece_id=QPNEP7GWFPHGOOGQDMYRCBLCY2O77362U4AHMCBQGITP44GVYKNA, seg_pos={0 0}, upload=220138]: Done uploading piece. err: ecclient: upload failed (node:12cnVqYYfo8b6RvRtMykzh8mU7UAqxcqYvWe7H8G7k8DqtNLAR4, address:104.36.82.22:4512): protocol: rpc: dial tcp 104.36.82.22:4512: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:02:47.429227805 (211h36m28.160495435s) [noise=true , address=104.36.82.34:4549 , piece_id=4XNPDUOWGVO24ZEWVAIHXV5BF4CMHPB6X6FACBKJ74SJUDEARWXQ, seg_pos={0 0}, upload=220138]: Done uploading piece. err: ecclient: upload failed (node:12tn1Sd8Gk3eHSmToBnPGjCyaAdQpUQrhiGe5cuecwBCpd7kEyT, address:104.36.82.34:4549): protocol: rpc: dial tcp 104.36.82.34:4549: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:02:47.429243795 (211h36m28.160511415s) [seg_pos={0 0}, upload=220138]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil> Aug 17 13:02:47.444337726 (211h36m28.175605347s) [upload=220138]: Done uploading segments... Aug 17 13:02:47.444359277 (211h36m28.175626896s) [upload=220138]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9) Duration: 1m0.01800382s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.026628661s FAIL (2025-08-17 13:02:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Aug 17 13:02:14.166843453 (211h35m54.898111104s) [upload=220146]: Starting upload Aug 17 13:02:14.166926810 (211h35m54.89819446s) [upload=220146]: Done starting upload Aug 17 13:02:14.167030756 (211h35m54.898298406s) [upload=220146]: Uploading segments... Aug 17 13:02:14.167323777 (211h35m54.898591426s) [upload=220146]: Got next segment. Inline: false Aug 17 13:02:14.167369473 (211h35m54.898637124s) [upload=220146]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:02:14.167375064 (211h35m54.898642714s) [upload=220146]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:02:14.195460783 (211h35m54.926728434s) [seg_pos={0 0}, upload=220146]: Begin upload segment... Aug 17 13:02:14.195505318 (211h35m54.926772968s) [seg_pos={0 0}, upload=220146]: Done begin upload segment. Aug 17 13:02:14.195537949 (211h35m54.926805599s) [noise=true , address=104.36.82.16:4293 , piece_id=4OJY43SQCNYECW4EVCM6IRJTXAEG32KIPNPJNHDA7U3SKFKXCXYA, seg_pos={0 0}, upload=220146]: Uploading piece... Aug 17 13:02:14.195551224 (211h35m54.926818875s) [noise=true , address=77.90.31.15:4699 , piece_id=JLCEJMHMWCFFLW4EMZMDMM552PL73QQVTWVLLCB4VYNZBWLA7L2Q, seg_pos={0 0}, upload=220146]: Uploading piece... Aug 17 13:02:14.196055232 (211h35m54.927322881s) [noise=true , address=104.36.82.11:4115 , piece_id=DNLPKM3REHYO7YME4RJNPIEDVIZUM5G2JSZW7ZRK3QMS3STVFI6A, seg_pos={0 0}, upload=220146]: Uploading piece... Aug 17 13:02:14.196195385 (211h35m54.927463035s) [noise=true , address=104.36.82.2:4729 , piece_id=X3DSGEAX4JCIYWJKZR5YZJZJDOUIKINDXOH2NEEQEUAISRWT4UZA, seg_pos={0 0}, upload=220146]: Uploading piece... Aug 17 13:02:14.196523953 (211h35m54.927791612s) [noise=true , address=45.91.116.5:4299 , piece_id=TAKWOO6SVUZO6WUNH66UCEWGRX4KJIAI2TYXQ2AE6AQPWSF3Y75A, seg_pos={0 0}, upload=220146]: Uploading piece... Aug 17 13:02:14.196636685 (211h35m54.927904335s) [noise=true , address=104.36.82.9:4007 , piece_id=3F5VMPHYHX44CKJLGGBWCX5VSOHJQRYGPWBODVLUECBICDT3RZ2A, seg_pos={0 0}, upload=220146]: Uploading piece... Aug 17 13:02:14.197040073 (211h35m54.928307724s) [noise=true , address=45.91.116.5:4297 , piece_id=M2UIYM6JB2MHGWNE5XGWXQONSBENYIFHBVILLSDKQZKNZIV46ECA, seg_pos={0 0}, upload=220146]: Uploading piece... Aug 17 13:02:14.197086951 (211h35m54.928354602s) [noise=true , address=45.91.116.4:4938 , piece_id=3PBELG7JEB6XFGPDSTFWHDDQXLEYOBXPRBAXJUD2ADUIL43EVPUA, seg_pos={0 0}, upload=220146]: Uploading piece... Aug 17 13:02:14.197601600 (211h35m54.92886925s) [noise=true , address=45.91.116.8:4045 , piece_id=R2QWS6ISMH2ZZ2CFXOO5S7ZEEOUJIO3FPJ7L6JYEJLUDSB7GKLEA, seg_pos={0 0}, upload=220146]: Uploading piece... Aug 17 13:02:14.198097241 (211h35m54.929364891s) [noise=true , address=45.91.116.7:4011 , piece_id=T7FE3F5EJ4674NOCIGZPFDUOQ6DZTOT2WYCY4POV5APEJDMKAK7A, seg_pos={0 0}, upload=220146]: Uploading piece... Aug 17 13:02:14.198561043 (211h35m54.929828693s) [noise=true , address=45.91.116.7:4009 , piece_id=GNDZSHLHVICOJZSD3IHS2IDNPM2RTWR5KB4JVJRDBFD6F3NIEB7A, seg_pos={0 0}, upload=220146]: Uploading piece... Aug 17 13:02:14.198916051 (211h35m54.930183711s) [noise=true , address=45.91.116.10:4081 , piece_id=7UAXKRETACXXEE6UITITSZA64H3BQA72ULRIAMQ7ZXFFJLZW4RAA, seg_pos={0 0}, upload=220146]: Uploading piece... Aug 17 13:02:14.199274856 (211h35m54.930542506s) [noise=true , address=45.91.116.13:4155 , piece_id=AJ3PG5NTEMNV6HYNLAIMQHTTLEVZFRZPANSJ3SEBMCYE6LGJRPRQ, seg_pos={0 0}, upload=220146]: Uploading piece... Aug 17 13:02:14.199639921 (211h35m54.930907572s) [noise=true , address=45.91.116.9:4263 , piece_id=2GNH6K4LLG556F7HVCFLYXJCY6SRHNFGKKSIWLBT4L2AG3OWXMXA, seg_pos={0 0}, upload=220146]: Uploading piece... Aug 17 13:02:14.200023723 (211h35m54.931291374s) [noise=true , address=45.91.116.14:4189 , piece_id=TG4NMHC3PBDFPBWOD3SOAJVYIIZO35ZM2EBEJDKBA2EWNRV7H73A, seg_pos={0 0}, upload=220146]: Uploading piece... Aug 17 13:02:14.200436069 (211h35m54.931703729s) [noise=true , address=45.91.116.11:4118 , piece_id=F562R4YRGIVS6RT5MMHSQ6EW4BUSZD7ELL3JATGZYUOYXIRE3FAA, seg_pos={0 0}, upload=220146]: Uploading piece... Aug 17 13:02:14.319016257 (211h35m55.050283917s) [upload=220146]: Next returned nil segment Aug 17 13:02:14.319031916 (211h35m55.050299566s) [upload=220146]: Waiting for error group managing segments... Aug 17 13:02:14.327246712 (211h35m55.058514362s) [noise=true , address=45.91.116.11:4118 , piece_id=F562R4YRGIVS6RT5MMHSQ6EW4BUSZD7ELL3JATGZYUOYXIRE3FAA, seg_pos={0 0}, upload=220146]: Done uploading piece. err: <nil> Aug 17 13:02:14.327874554 (211h35m55.059142214s) [noise=true , address=45.91.116.14:4189 , piece_id=TG4NMHC3PBDFPBWOD3SOAJVYIIZO35ZM2EBEJDKBA2EWNRV7H73A, seg_pos={0 0}, upload=220146]: Done uploading piece. err: <nil> Aug 17 13:02:14.328100819 (211h35m55.059368469s) [noise=true , address=45.91.116.4:4938 , piece_id=3PBELG7JEB6XFGPDSTFWHDDQXLEYOBXPRBAXJUD2ADUIL43EVPUA, seg_pos={0 0}, upload=220146]: Done uploading piece. err: <nil> Aug 17 13:02:14.328775447 (211h35m55.060043098s) [noise=true , address=45.91.116.8:4045 , piece_id=R2QWS6ISMH2ZZ2CFXOO5S7ZEEOUJIO3FPJ7L6JYEJLUDSB7GKLEA, seg_pos={0 0}, upload=220146]: Done uploading piece. err: <nil> Aug 17 13:02:14.328900151 (211h35m55.060167801s) [noise=true , address=45.91.116.5:4297 , piece_id=M2UIYM6JB2MHGWNE5XGWXQONSBENYIFHBVILLSDKQZKNZIV46ECA, seg_pos={0 0}, upload=220146]: Done uploading piece. err: <nil> Aug 17 13:02:14.329506932 (211h35m55.060774582s) [noise=true , address=45.91.116.7:4009 , piece_id=GNDZSHLHVICOJZSD3IHS2IDNPM2RTWR5KB4JVJRDBFD6F3NIEB7A, seg_pos={0 0}, upload=220146]: Done uploading piece. err: <nil> Aug 17 13:02:14.329598064 (211h35m55.060865725s) [noise=true , address=45.91.116.5:4299 , piece_id=TAKWOO6SVUZO6WUNH66UCEWGRX4KJIAI2TYXQ2AE6AQPWSF3Y75A, seg_pos={0 0}, upload=220146]: Done uploading piece. err: <nil> Aug 17 13:02:14.330067166 (211h35m55.061334816s) [noise=true , address=45.91.116.9:4263 , piece_id=2GNH6K4LLG556F7HVCFLYXJCY6SRHNFGKKSIWLBT4L2AG3OWXMXA, seg_pos={0 0}, upload=220146]: Done uploading piece. err: <nil> Aug 17 13:02:14.330183445 (211h35m55.061451095s) [noise=true , address=45.91.116.10:4081 , piece_id=7UAXKRETACXXEE6UITITSZA64H3BQA72ULRIAMQ7ZXFFJLZW4RAA, seg_pos={0 0}, upload=220146]: Done uploading piece. err: <nil> Aug 17 13:02:14.330615978 (211h35m55.061883628s) [noise=true , address=45.91.116.7:4011 , piece_id=T7FE3F5EJ4674NOCIGZPFDUOQ6DZTOT2WYCY4POV5APEJDMKAK7A, seg_pos={0 0}, upload=220146]: Done uploading piece. err: <nil> Aug 17 13:02:14.330720905 (211h35m55.061988555s) [noise=true , address=45.91.116.13:4155 , piece_id=AJ3PG5NTEMNV6HYNLAIMQHTTLEVZFRZPANSJ3SEBMCYE6LGJRPRQ, seg_pos={0 0}, upload=220146]: Done uploading piece. err: <nil> Aug 17 13:02:14.359481354 (211h35m55.090749005s) [noise=true , address=77.90.31.15:4699 , piece_id=JLCEJMHMWCFFLW4EMZMDMM552PL73QQVTWVLLCB4VYNZBWLA7L2Q, seg_pos={0 0}, upload=220146]: Done uploading piece. err: <nil> Aug 17 13:02:44.153915446 (211h36m24.885183066s) [noise=true , address=104.36.82.11:4115 , piece_id=DNLPKM3REHYO7YME4RJNPIEDVIZUM5G2JSZW7ZRK3QMS3STVFI6A, seg_pos={0 0}, upload=220146]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: rpc: dial tcp 104.36.82.11:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:02:44.153964557 (211h36m24.885232178s) [noise=true , address=104.36.82.9:4007 , piece_id=3F5VMPHYHX44CKJLGGBWCX5VSOHJQRYGPWBODVLUECBICDT3RZ2A, seg_pos={0 0}, upload=220146]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 5.161.89.252:22116->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:02:44.153997319 (211h36m24.885264949s) [noise=true , address=104.36.82.2:4729 , piece_id=X3DSGEAX4JCIYWJKZR5YZJZJDOUIKINDXOH2NEEQEUAISRWT4UZA, seg_pos={0 0}, upload=220146]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: rpc: dial tcp 104.36.82.2:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:02:44.154037426 (211h36m24.885305075s) [noise=true , address=104.36.82.16:4293 , piece_id=4OJY43SQCNYECW4EVCM6IRJTXAEG32KIPNPJNHDA7U3SKFKXCXYA, seg_pos={0 0}, upload=220146]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:104.36.82.16:4293): protocol: write tcp 5.161.89.252:52496->104.36.82.16:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:02:44.154077601 (211h36m24.885345231s) [seg_pos={0 0}, upload=220146]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Aug 17 13:02:44.178328801 (211h36m24.909596441s) [upload=220146]: Done uploading segments... Aug 17 13:02:44.178348317 (211h36m24.909615957s) [upload=220146]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.026648107s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.01872012s FAIL (2025-08-17 13:02:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:01:39.081086304 (211h35m19.812353934s) [upload=220136]: Starting upload Aug 17 13:01:39.081157999 (211h35m19.812425639s) [upload=220136]: Done starting upload Aug 17 13:01:39.081309654 (211h35m19.812577284s) [upload=220136]: Uploading segments... Aug 17 13:01:39.081425782 (211h35m19.812693401s) [upload=220136]: Got next segment. Inline: false Aug 17 13:01:39.081470596 (211h35m19.812738226s) [upload=220136]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:01:39.081474703 (211h35m19.812742323s) [upload=220136]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:01:39.101614932 (211h35m19.832882562s) [seg_pos={0 0}, upload=220136]: Begin upload segment... Aug 17 13:01:39.101666360 (211h35m19.83293399s) [seg_pos={0 0}, upload=220136]: Done begin upload segment. Aug 17 13:01:39.101700003 (211h35m19.832967633s) [noise=true , address=45.91.116.10:4081 , piece_id=YORNG6ZYZLT44TKJS3CP4UOZUE6DR6RNDNIT2HDMY7Y5AMO25RLA, seg_pos={0 0}, upload=220136]: Uploading piece... Aug 17 13:01:39.101710894 (211h35m19.832978514s) [noise=true , address=104.36.82.2:4730 , piece_id=X4APLCYNI4TELPSYW2MYE5ROTPM2SRV2VGULWZJK6KHCXHYJHROA, seg_pos={0 0}, upload=220136]: Uploading piece... Aug 17 13:01:39.102062294 (211h35m19.833329914s) [noise=true , address=104.36.82.8:4969 , piece_id=OZPYE3AYPSNOBTLEAFWWZ35SSEDCKTZL262H7JW37EZT4WY5VIYQ, seg_pos={0 0}, upload=220136]: Uploading piece... Aug 17 13:01:39.102073936 (211h35m19.833341567s) [noise=true , address=77.90.31.21:4877 , piece_id=D5ITQ5R3HOLTTZE354G2UAJBQKMGFJUCGNFM7SR7QXOMNRWKCJ4A, seg_pos={0 0}, upload=220136]: Uploading piece... Aug 17 13:01:39.102347450 (211h35m19.833615081s) [noise=true , address=104.36.82.15:4257 , piece_id=ZBQZAMFG2N6NXT2GUPHTTLBRXUXK76P3ZMHPJX6JDJXXMFDGBJFQ, seg_pos={0 0}, upload=220136]: Uploading piece... Aug 17 13:01:39.102354674 (211h35m19.833622294s) [noise=true , address=45.91.116.5:4298 , piece_id=REYD6SNCN57HRHR3TV2VYUD4EMRIFVBJPBWCVSCC3ZTFXTEXTYYA, seg_pos={0 0}, upload=220136]: Uploading piece... Aug 17 13:01:39.102605425 (211h35m19.833873055s) [noise=true , address=104.36.82.6:4875 , piece_id=5ZR6EQCDULCZHSUWAWOX4YKUQBSZJ5TIG754EZ5RX6XUWT5ICN3A, seg_pos={0 0}, upload=220136]: Uploading piece... Aug 17 13:01:39.102686568 (211h35m19.833954199s) [noise=true , address=77.90.31.13:4628 , piece_id=3O7VRYRY4L6RZV4XIUESHQN4VTJM3NU2CACMPOKNV645VDE3D3SA, seg_pos={0 0}, upload=220136]: Uploading piece... Aug 17 13:01:39.102843963 (211h35m19.834111593s) [noise=true , address=104.36.82.11:4114 , piece_id=37OGBPUX5WAPDXETSYBEYFH6Q6O7C7LUHGFTQRZH6CO62GGLLJWQ, seg_pos={0 0}, upload=220136]: Uploading piece... Aug 17 13:01:39.102964110 (211h35m19.83423173s) [noise=true , address=45.91.116.9:4262 , piece_id=GYNTQQESAQRLIKLSJ4EUVSUA5CJQV2OUHTTLUKZMPV2JAIED2OBA, seg_pos={0 0}, upload=220136]: Uploading piece... Aug 17 13:01:39.103081640 (211h35m19.83434927s) [noise=true , address=104.36.82.12:4151 , piece_id=Y2EOGNHIIQTVSVGBCSAFNH3CAHUQI5NDIZZYRYAUT26A2OPRRS3Q, seg_pos={0 0}, upload=220136]: Uploading piece... Aug 17 13:01:39.103204441 (211h35m19.834472062s) [noise=true , address=45.91.116.10:4083 , piece_id=SDTUFC7HVNIU5YLFLNLPM76F5JOKZON542LC4XMJRFHER2XOHXNQ, seg_pos={0 0}, upload=220136]: Uploading piece... Aug 17 13:01:39.103332864 (211h35m19.834600484s) [noise=true , address=77.90.31.18:4807 , piece_id=4H7CJCS4KZFMAK4LX22WYMXE677WWCG3SXFBUSBMYKXF7TI6AJGA, seg_pos={0 0}, upload=220136]: Uploading piece... Aug 17 13:01:39.103567354 (211h35m19.834834984s) [noise=true , address=77.90.31.11:4553 , piece_id=J7K4ZVEC44NMBF6IILG5JF4363S6XFC2BPVVEZJCCEOXO36K3NHA, seg_pos={0 0}, upload=220136]: Uploading piece... Aug 17 13:01:39.103790113 (211h35m19.835057733s) [noise=true , address=45.91.116.12:4227 , piece_id=DYFFBY3N64YTIHJZWYXTKR2TETNIJR2YJNG3OOMM3DTRB4WLIVLA, seg_pos={0 0}, upload=220136]: Uploading piece... Aug 17 13:01:39.104145721 (211h35m19.835413341s) [noise=true , address=104.36.82.14:4223 , piece_id=2RK6ZCLDG4BCBN72KHJRRCJB6HNHF7VHTSELYSAY376SZHKWLITQ, seg_pos={0 0}, upload=220136]: Uploading piece... Aug 17 13:01:39.197047955 (211h35m19.928315576s) [upload=220136]: Next returned nil segment Aug 17 13:01:39.197095265 (211h35m19.928362875s) [upload=220136]: Waiting for error group managing segments... Aug 17 13:01:39.205834287 (211h35m19.937101907s) [noise=true , address=45.91.116.5:4298 , piece_id=REYD6SNCN57HRHR3TV2VYUD4EMRIFVBJPBWCVSCC3ZTFXTEXTYYA, seg_pos={0 0}, upload=220136]: Done uploading piece. err: <nil> Aug 17 13:01:39.206553188 (211h35m19.937820809s) [noise=true , address=45.91.116.9:4262 , piece_id=GYNTQQESAQRLIKLSJ4EUVSUA5CJQV2OUHTTLUKZMPV2JAIED2OBA, seg_pos={0 0}, upload=220136]: Done uploading piece. err: <nil> Aug 17 13:01:39.206993687 (211h35m19.938261306s) [noise=true , address=45.91.116.10:4083 , piece_id=SDTUFC7HVNIU5YLFLNLPM76F5JOKZON542LC4XMJRFHER2XOHXNQ, seg_pos={0 0}, upload=220136]: Done uploading piece. err: <nil> Aug 17 13:01:39.207051616 (211h35m19.938319246s) [noise=true , address=45.91.116.10:4081 , piece_id=YORNG6ZYZLT44TKJS3CP4UOZUE6DR6RNDNIT2HDMY7Y5AMO25RLA, seg_pos={0 0}, upload=220136]: Done uploading piece. err: <nil> Aug 17 13:01:39.207394911 (211h35m19.938662531s) [noise=true , address=77.90.31.13:4628 , piece_id=3O7VRYRY4L6RZV4XIUESHQN4VTJM3NU2CACMPOKNV645VDE3D3SA, seg_pos={0 0}, upload=220136]: Done uploading piece. err: <nil> Aug 17 13:01:39.212470706 (211h35m19.943738336s) [noise=true , address=77.90.31.11:4553 , piece_id=J7K4ZVEC44NMBF6IILG5JF4363S6XFC2BPVVEZJCCEOXO36K3NHA, seg_pos={0 0}, upload=220136]: Done uploading piece. err: <nil> Aug 17 13:01:39.229513211 (211h35m19.960780831s) [noise=true , address=77.90.31.21:4877 , piece_id=D5ITQ5R3HOLTTZE354G2UAJBQKMGFJUCGNFM7SR7QXOMNRWKCJ4A, seg_pos={0 0}, upload=220136]: Done uploading piece. err: <nil> Aug 17 13:01:39.244137018 (211h35m19.975404648s) [noise=true , address=77.90.31.18:4807 , piece_id=4H7CJCS4KZFMAK4LX22WYMXE677WWCG3SXFBUSBMYKXF7TI6AJGA, seg_pos={0 0}, upload=220136]: Done uploading piece. err: <nil> Aug 17 13:01:39.244894322 (211h35m19.976161943s) [noise=true , address=45.91.116.12:4227 , piece_id=DYFFBY3N64YTIHJZWYXTKR2TETNIJR2YJNG3OOMM3DTRB4WLIVLA, seg_pos={0 0}, upload=220136]: Done uploading piece. err: <nil> Aug 17 13:02:09.064067523 (211h35m49.795335143s) [noise=true , address=104.36.82.8:4969 , piece_id=OZPYE3AYPSNOBTLEAFWWZ35SSEDCKTZL262H7JW37EZT4WY5VIYQ, seg_pos={0 0}, upload=220136]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: expected piece hash; context deadline exceeded; EOF Aug 17 13:02:09.064081839 (211h35m49.795349469s) [noise=true , address=104.36.82.2:4730 , piece_id=X4APLCYNI4TELPSYW2MYE5ROTPM2SRV2VGULWZJK6KHCXHYJHROA, seg_pos={0 0}, upload=220136]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: expected piece hash; context deadline exceeded; EOF Aug 17 13:02:09.064126904 (211h35m49.795394534s) [noise=true , address=104.36.82.15:4257 , piece_id=ZBQZAMFG2N6NXT2GUPHTTLBRXUXK76P3ZMHPJX6JDJXXMFDGBJFQ, seg_pos={0 0}, upload=220136]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: expected piece hash; context deadline exceeded; EOF Aug 17 13:02:09.064162381 (211h35m49.795430011s) [noise=true , address=104.36.82.12:4151 , piece_id=Y2EOGNHIIQTVSVGBCSAFNH3CAHUQI5NDIZZYRYAUT26A2OPRRS3Q, seg_pos={0 0}, upload=220136]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: expected piece hash; context deadline exceeded; EOF Aug 17 13:02:09.064177038 (211h35m49.795444659s) [noise=true , address=104.36.82.14:4223 , piece_id=2RK6ZCLDG4BCBN72KHJRRCJB6HNHF7VHTSELYSAY376SZHKWLITQ, seg_pos={0 0}, upload=220136]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: expected piece hash; context deadline exceeded; EOF Aug 17 13:02:09.064183901 (211h35m49.795451521s) [noise=true , address=104.36.82.11:4114 , piece_id=37OGBPUX5WAPDXETSYBEYFH6Q6O7C7LUHGFTQRZH6CO62GGLLJWQ, seg_pos={0 0}, upload=220136]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: expected piece hash; context deadline exceeded; EOF Aug 17 13:02:09.064190393 (211h35m49.795458013s) [noise=true , address=104.36.82.6:4875 , piece_id=5ZR6EQCDULCZHSUWAWOX4YKUQBSZJ5TIG754EZ5RX6XUWT5ICN3A, seg_pos={0 0}, upload=220136]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:104.36.82.6:4875): protocol: expected piece hash; context deadline exceeded; EOF Aug 17 13:02:09.064212265 (211h35m49.795479895s) [seg_pos={0 0}, upload=220136]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:02:09.081672858 (211h35m49.812940488s) [upload=220136]: Done uploading segments... Aug 17 13:02:09.081685481 (211h35m49.812953111s) [upload=220136]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.018737243s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X..............X..] duration: 356.62679ms FAIL (2025-08-17 13:01:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 356.645444ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [X..........................................X..............X.] duration: 1.316963891s FAIL (2025-08-17 13:01:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.316986003s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X................................................X...] duration: 469.42585ms FAIL (2025-08-17 13:01:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 12:58:50.860477437 (211h32m31.591745088s) [upload=220080]: Starting upload Aug 17 12:58:50.860545436 (211h32m31.591813087s) [upload=220080]: Done starting upload Aug 17 12:58:50.860609366 (211h32m31.591877016s) [upload=220080]: Uploading segments... Aug 17 12:58:50.860948534 (211h32m31.592216184s) [upload=220080]: Got next segment. Inline: false Aug 17 12:58:50.860999970 (211h32m31.592267621s) [upload=220080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 12:58:50.861006203 (211h32m31.592273852s) [upload=220080]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 12:58:50.865191131 (211h32m31.596458782s) [upload=220080]: Next segment err: metaclient: Too Many Requests Aug 17 12:58:50.865207953 (211h32m31.596475613s) [upload=220080]: Done uploading segments... Aug 17 12:58:50.865221468 (211h32m31.596489118s) [upload=220080]: Upload finished. err: metaclient: Too Many Requests Duration: 469.453302ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.X..........................................X..............X] duration: 16.305821ms FAIL (2025-08-17 13:00:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 16.325547ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [........X................................................X..] duration: 628.384482ms FAIL (2025-08-17 12:59:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 12:58:50.860477437 (211h32m31.591745088s) [upload=220080]: Starting upload Aug 17 12:58:50.860545436 (211h32m31.591813087s) [upload=220080]: Done starting upload Aug 17 12:58:50.860609366 (211h32m31.591877016s) [upload=220080]: Uploading segments... Aug 17 12:58:50.860948534 (211h32m31.592216184s) [upload=220080]: Got next segment. Inline: false Aug 17 12:58:50.860999970 (211h32m31.592267621s) [upload=220080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 12:58:50.861006203 (211h32m31.592273852s) [upload=220080]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 12:58:50.865191131 (211h32m31.596458782s) [upload=220080]: Next segment err: metaclient: Too Many Requests Aug 17 12:58:50.865207953 (211h32m31.596475613s) [upload=220080]: Done uploading segments... Aug 17 12:58:50.865221468 (211h32m31.596489118s) [upload=220080]: Upload finished. err: metaclient: Too Many Requests Duration: 628.4099ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..X..........................................X..............] duration: 137.577208ms FAIL (2025-08-17 12:59:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 12:49:34.130817816 (211h23m14.862085466s) [upload=219921]: Starting upload Aug 17 12:49:34.130874141 (211h23m14.862141802s) [upload=219921]: Done starting upload Aug 17 12:49:34.130935236 (211h23m14.862202886s) [upload=219921]: Uploading segments... Aug 17 12:49:34.130974540 (211h23m14.86224219s) [upload=219921]: Got next segment. Inline: true Aug 17 12:49:34.130979569 (211h23m14.86224722s) [upload=219921]: Waiting for error group managing segments... Aug 17 12:49:34.131023363 (211h23m14.862291012s) [upload=219921]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 12:49:34.131028712 (211h23m14.862296373s) [upload=219921]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 12:49:34.131033561 (211h23m14.862301212s) [upload=219921]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 12:49:34.133605427 (211h23m14.864873088s) [upload=219921]: Done uploading segments... Aug 17 12:49:34.133617179 (211h23m14.86488483s) [upload=219921]: Upload finished. err: metaclient: Too Many Requests Duration:* 137.595803ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...X..........................................X.............] duration: 283.51101ms FAIL (2025-08-17 12:59:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 12:49:34.130817816 (211h23m14.862085466s) [upload=219921]: Starting upload Aug 17 12:49:34.130874141 (211h23m14.862141802s) [upload=219921]: Done starting upload Aug 17 12:49:34.130935236 (211h23m14.862202886s) [upload=219921]: Uploading segments... Aug 17 12:49:34.130974540 (211h23m14.86224219s) [upload=219921]: Got next segment. Inline: true Aug 17 12:49:34.130979569 (211h23m14.86224722s) [upload=219921]: Waiting for error group managing segments... Aug 17 12:49:34.131023363 (211h23m14.862291012s) [upload=219921]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 12:49:34.131028712 (211h23m14.862296373s) [upload=219921]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 12:49:34.131033561 (211h23m14.862301212s) [upload=219921]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 12:49:34.133605427 (211h23m14.864873088s) [upload=219921]: Done uploading segments... Aug 17 12:49:34.133617179 (211h23m14.86488483s) [upload=219921]: Upload finished. err: metaclient: Too Many Requests Duration:* 283.580742ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........X................................................X.] duration: 801.696504ms FAIL (2025-08-17 12:59:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 12:58:50.860477437 (211h32m31.591745088s) [upload=220080]: Starting upload Aug 17 12:58:50.860545436 (211h32m31.591813087s) [upload=220080]: Done starting upload Aug 17 12:58:50.860609366 (211h32m31.591877016s) [upload=220080]: Uploading segments... Aug 17 12:58:50.860948534 (211h32m31.592216184s) [upload=220080]: Got next segment. Inline: false Aug 17 12:58:50.860999970 (211h32m31.592267621s) [upload=220080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 12:58:50.861006203 (211h32m31.592273852s) [upload=220080]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 12:58:50.865191131 (211h32m31.596458782s) [upload=220080]: Next segment err: metaclient: Too Many Requests Aug 17 12:58:50.865207953 (211h32m31.596475613s) [upload=220080]: Done uploading segments... Aug 17 12:58:50.865221468 (211h32m31.596489118s) [upload=220080]: Upload finished. err: metaclient: Too Many Requests Duration: 801.717623ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........X................................................X] duration: 26.595437ms FAIL (2025-08-17 12:58:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 12:58:50.860477437 (211h32m31.591745088s) [upload=220080]: Starting upload Aug 17 12:58:50.860545436 (211h32m31.591813087s) [upload=220080]: Done starting upload Aug 17 12:58:50.860609366 (211h32m31.591877016s) [upload=220080]: Uploading segments... Aug 17 12:58:50.860948534 (211h32m31.592216184s) [upload=220080]: Got next segment. Inline: false Aug 17 12:58:50.860999970 (211h32m31.592267621s) [upload=220080]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 12:58:50.861006203 (211h32m31.592273852s) [upload=220080]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 12:58:50.865191131 (211h32m31.596458782s) [upload=220080]: Next segment err: metaclient: Too Many Requests Aug 17 12:58:50.865207953 (211h32m31.596475613s) [upload=220080]: Done uploading segments... Aug 17 12:58:50.865221468 (211h32m31.596489118s) [upload=220080]: Upload finished. err: metaclient: Too Many Requests Duration: 26.615164ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....X..........................................X............] duration: 149.352921ms FAIL (2025-08-17 12:58:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 12:49:34.130817816 (211h23m14.862085466s) [upload=219921]: Starting upload Aug 17 12:49:34.130874141 (211h23m14.862141802s) [upload=219921]: Done starting upload Aug 17 12:49:34.130935236 (211h23m14.862202886s) [upload=219921]: Uploading segments... Aug 17 12:49:34.130974540 (211h23m14.86224219s) [upload=219921]: Got next segment. Inline: true Aug 17 12:49:34.130979569 (211h23m14.86224722s) [upload=219921]: Waiting for error group managing segments... Aug 17 12:49:34.131023363 (211h23m14.862291012s) [upload=219921]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 12:49:34.131028712 (211h23m14.862296373s) [upload=219921]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 12:49:34.131033561 (211h23m14.862301212s) [upload=219921]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 12:49:34.133605427 (211h23m14.864873088s) [upload=219921]: Done uploading segments... Aug 17 12:49:34.133617179 (211h23m14.86488483s) [upload=219921]: Upload finished. err: metaclient: Too Many Requests Duration:* 149.370564ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....X..........................................X...........] duration: 169.052843ms FAIL (2025-08-17 12:57:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 12:49:34.130817816 (211h23m14.862085466s) [upload=219921]: Starting upload Aug 17 12:49:34.130874141 (211h23m14.862141802s) [upload=219921]: Done starting upload Aug 17 12:49:34.130935236 (211h23m14.862202886s) [upload=219921]: Uploading segments... Aug 17 12:49:34.130974540 (211h23m14.86224219s) [upload=219921]: Got next segment. Inline: true Aug 17 12:49:34.130979569 (211h23m14.86224722s) [upload=219921]: Waiting for error group managing segments... Aug 17 12:49:34.131023363 (211h23m14.862291012s) [upload=219921]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 12:49:34.131028712 (211h23m14.862296373s) [upload=219921]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 12:49:34.131033561 (211h23m14.862301212s) [upload=219921]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 12:49:34.133605427 (211h23m14.864873088s) [upload=219921]: Done uploading segments... Aug 17 12:49:34.133617179 (211h23m14.86488483s) [upload=219921]: Upload finished. err: metaclient: Too Many Requests Duration:* 169.078682ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 610.928198ms FAIL (2025-08-17 12:57:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 12:20:16.771641580 (210h53m57.50290923s) [upload=219410]: Starting upload Aug 17 12:20:16.771838160 (210h53m57.503105811s) [upload=219410]: Done starting upload Aug 17 12:20:16.772244565 (210h53m57.503512215s) [upload=219410]: Uploading segments... Aug 17 12:20:16.772504924 (210h53m57.503772634s) [upload=219410]: Got next segment. Inline: false Aug 17 12:20:16.772558475 (210h53m57.503826126s) [upload=219410]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 12:20:16.772566721 (210h53m57.503834381s) [upload=219410]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 12:20:16.775373039 (210h53m57.50664069s) [upload=219410]: Next segment err: metaclient: Too Many Requests Aug 17 12:20:16.775393207 (210h53m57.506660857s) [upload=219410]: Done uploading segments... Aug 17 12:20:16.775413586 (210h53m57.506681287s) [upload=219410]: Upload finished. err: metaclient: Too Many Requests Duration: 610.970147ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 441.594778ms FAIL (2025-08-17 12:57:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 12:20:16.771641580 (210h53m57.50290923s) [upload=219410]: Starting upload Aug 17 12:20:16.771838160 (210h53m57.503105811s) [upload=219410]: Done starting upload Aug 17 12:20:16.772244565 (210h53m57.503512215s) [upload=219410]: Uploading segments... Aug 17 12:20:16.772504924 (210h53m57.503772634s) [upload=219410]: Got next segment. Inline: false Aug 17 12:20:16.772558475 (210h53m57.503826126s) [upload=219410]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 12:20:16.772566721 (210h53m57.503834381s) [upload=219410]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 12:20:16.775373039 (210h53m57.50664069s) [upload=219410]: Next segment err: metaclient: Too Many Requests Aug 17 12:20:16.775393207 (210h53m57.506660857s) [upload=219410]: Done uploading segments... Aug 17 12:20:16.775413586 (210h53m57.506681287s) [upload=219410]: Upload finished. err: metaclient: Too Many Requests Duration: 441.622831ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [......X..........................................X..........] duration: 151.634892ms FAIL (2025-08-17 12:56:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 12:49:34.130817816 (211h23m14.862085466s) [upload=219921]: Starting upload Aug 17 12:49:34.130874141 (211h23m14.862141802s) [upload=219921]: Done starting upload Aug 17 12:49:34.130935236 (211h23m14.862202886s) [upload=219921]: Uploading segments... Aug 17 12:49:34.130974540 (211h23m14.86224219s) [upload=219921]: Got next segment. Inline: true Aug 17 12:49:34.130979569 (211h23m14.86224722s) [upload=219921]: Waiting for error group managing segments... Aug 17 12:49:34.131023363 (211h23m14.862291012s) [upload=219921]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 12:49:34.131028712 (211h23m14.862296373s) [upload=219921]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 12:49:34.131033561 (211h23m14.862301212s) [upload=219921]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 12:49:34.133605427 (211h23m14.864873088s) [upload=219921]: Done uploading segments... Aug 17 12:49:34.133617179 (211h23m14.86488483s) [upload=219921]: Upload finished. err: metaclient: Too Many Requests Duration:* 151.648477ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......X..........................................X.........] duration: 145.724236ms FAIL (2025-08-17 12:56:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 12:49:34.130817816 (211h23m14.862085466s) [upload=219921]: Starting upload Aug 17 12:49:34.130874141 (211h23m14.862141802s) [upload=219921]: Done starting upload Aug 17 12:49:34.130935236 (211h23m14.862202886s) [upload=219921]: Uploading segments... Aug 17 12:49:34.130974540 (211h23m14.86224219s) [upload=219921]: Got next segment. Inline: true Aug 17 12:49:34.130979569 (211h23m14.86224722s) [upload=219921]: Waiting for error group managing segments... Aug 17 12:49:34.131023363 (211h23m14.862291012s) [upload=219921]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 12:49:34.131028712 (211h23m14.862296373s) [upload=219921]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 12:49:34.131033561 (211h23m14.862301212s) [upload=219921]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 12:49:34.133605427 (211h23m14.864873088s) [upload=219921]: Done uploading segments... Aug 17 12:49:34.133617179 (211h23m14.86488483s) [upload=219921]: Upload finished. err: metaclient: Too Many Requests Duration:* 145.749593ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 623.077564ms FAIL (2025-08-17 12:56:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 12:20:16.771641580 (210h53m57.50290923s) [upload=219410]: Starting upload Aug 17 12:20:16.771838160 (210h53m57.503105811s) [upload=219410]: Done starting upload Aug 17 12:20:16.772244565 (210h53m57.503512215s) [upload=219410]: Uploading segments... Aug 17 12:20:16.772504924 (210h53m57.503772634s) [upload=219410]: Got next segment. Inline: false Aug 17 12:20:16.772558475 (210h53m57.503826126s) [upload=219410]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 12:20:16.772566721 (210h53m57.503834381s) [upload=219410]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 12:20:16.775373039 (210h53m57.50664069s) [upload=219410]: Next segment err: metaclient: Too Many Requests Aug 17 12:20:16.775393207 (210h53m57.506660857s) [upload=219410]: Done uploading segments... Aug 17 12:20:16.775413586 (210h53m57.506681287s) [upload=219410]: Upload finished. err: metaclient: Too Many Requests Duration: 623.102471ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........X..........................................X........] duration: 236.602397ms FAIL (2025-08-17 12:55:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 12:49:34.130817816 (211h23m14.862085466s) [upload=219921]: Starting upload Aug 17 12:49:34.130874141 (211h23m14.862141802s) [upload=219921]: Done starting upload Aug 17 12:49:34.130935236 (211h23m14.862202886s) [upload=219921]: Uploading segments... Aug 17 12:49:34.130974540 (211h23m14.86224219s) [upload=219921]: Got next segment. Inline: true Aug 17 12:49:34.130979569 (211h23m14.86224722s) [upload=219921]: Waiting for error group managing segments... Aug 17 12:49:34.131023363 (211h23m14.862291012s) [upload=219921]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 12:49:34.131028712 (211h23m14.862296373s) [upload=219921]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 12:49:34.131033561 (211h23m14.862301212s) [upload=219921]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 12:49:34.133605427 (211h23m14.864873088s) [upload=219921]: Done uploading segments... Aug 17 12:49:34.133617179 (211h23m14.86488483s) [upload=219921]: Upload finished. err: metaclient: Too Many Requests Duration:* 236.653685ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 488.068385ms FAIL (2025-08-17 12:55:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 12:20:16.771641580 (210h53m57.50290923s) [upload=219410]: Starting upload Aug 17 12:20:16.771838160 (210h53m57.503105811s) [upload=219410]: Done starting upload Aug 17 12:20:16.772244565 (210h53m57.503512215s) [upload=219410]: Uploading segments... Aug 17 12:20:16.772504924 (210h53m57.503772634s) [upload=219410]: Got next segment. Inline: false Aug 17 12:20:16.772558475 (210h53m57.503826126s) [upload=219410]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 12:20:16.772566721 (210h53m57.503834381s) [upload=219410]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 12:20:16.775373039 (210h53m57.50664069s) [upload=219410]: Next segment err: metaclient: Too Many Requests Aug 17 12:20:16.775393207 (210h53m57.506660857s) [upload=219410]: Done uploading segments... Aug 17 12:20:16.775413586 (210h53m57.506681287s) [upload=219410]: Upload finished. err: metaclient: Too Many Requests Duration: 488.123527ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 472.025669ms FAIL (2025-08-17 12:54:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 12:20:16.771641580 (210h53m57.50290923s) [upload=219410]: Starting upload Aug 17 12:20:16.771838160 (210h53m57.503105811s) [upload=219410]: Done starting upload Aug 17 12:20:16.772244565 (210h53m57.503512215s) [upload=219410]: Uploading segments... Aug 17 12:20:16.772504924 (210h53m57.503772634s) [upload=219410]: Got next segment. Inline: false Aug 17 12:20:16.772558475 (210h53m57.503826126s) [upload=219410]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 12:20:16.772566721 (210h53m57.503834381s) [upload=219410]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 12:20:16.775373039 (210h53m57.50664069s) [upload=219410]: Next segment err: metaclient: Too Many Requests Aug 17 12:20:16.775393207 (210h53m57.506660857s) [upload=219410]: Done uploading segments... Aug 17 12:20:16.775413586 (210h53m57.506681287s) [upload=219410]: Upload finished. err: metaclient: Too Many Requests Duration: 472.05812ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 461.385299ms FAIL (2025-08-17 12:54:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 12:20:16.771641580 (210h53m57.50290923s) [upload=219410]: Starting upload Aug 17 12:20:16.771838160 (210h53m57.503105811s) [upload=219410]: Done starting upload Aug 17 12:20:16.772244565 (210h53m57.503512215s) [upload=219410]: Uploading segments... Aug 17 12:20:16.772504924 (210h53m57.503772634s) [upload=219410]: Got next segment. Inline: false Aug 17 12:20:16.772558475 (210h53m57.503826126s) [upload=219410]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 12:20:16.772566721 (210h53m57.503834381s) [upload=219410]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 12:20:16.775373039 (210h53m57.50664069s) [upload=219410]: Next segment err: metaclient: Too Many Requests Aug 17 12:20:16.775393207 (210h53m57.506660857s) [upload=219410]: Done uploading segments... Aug 17 12:20:16.775413586 (210h53m57.506681287s) [upload=219410]: Upload finished. err: metaclient: Too Many Requests Duration: 461.400909ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........X..........................................X.......] duration: 143.797965ms FAIL (2025-08-17 12:54:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 12:49:34.130817816 (211h23m14.862085466s) [upload=219921]: Starting upload Aug 17 12:49:34.130874141 (211h23m14.862141802s) [upload=219921]: Done starting upload Aug 17 12:49:34.130935236 (211h23m14.862202886s) [upload=219921]: Uploading segments... Aug 17 12:49:34.130974540 (211h23m14.86224219s) [upload=219921]: Got next segment. Inline: true Aug 17 12:49:34.130979569 (211h23m14.86224722s) [upload=219921]: Waiting for error group managing segments... Aug 17 12:49:34.131023363 (211h23m14.862291012s) [upload=219921]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 12:49:34.131028712 (211h23m14.862296373s) [upload=219921]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 12:49:34.131033561 (211h23m14.862301212s) [upload=219921]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 12:49:34.133605427 (211h23m14.864873088s) [upload=219921]: Done uploading segments... Aug 17 12:49:34.133617179 (211h23m14.86488483s) [upload=219921]: Upload finished. err: metaclient: Too Many Requests Duration:* 143.819084ms core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)