Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..............................X.X.........X.......X...X.....] duration: 573.845872ms FAIL (2025-09-22 02:31:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 573.942793ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [..............................X.XXXXXX..X...................] duration: 441.79547ms FAIL (2025-09-22 02:31:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== m30.954857157s) [seg_pos={0 0}, upload=496461]: Begin upload segment... Sep 22 02:15:55.602116061 (564h7m30.954901189s) [seg_pos={0 0}, upload=496461]: Done begin upload segment. Sep 22 02:15:55.602137471 (564h7m30.954922599s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602154613 (564h7m30.954939742s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602716609 (564h7m30.955501736s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602736295 (564h7m30.955521423s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603134873 (564h7m30.955920001s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603640462 (564h7m30.95642559s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604036745 (564h7m30.956821874s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604397323 (564h7m30.95718245s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604828131 (564h7m30.957613268s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605250674 (564h7m30.958035802s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605663548 (564h7m30.958448676s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606022221 (564h7m30.958807349s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606387297 (564h7m30.959172425s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606869060 (564h7m30.959654198s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607319626 (564h7m30.960104755s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607717662 (564h7m30.960502801s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608211139 (564h7m30.960996268s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608652857 (564h7m30.961437986s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.704831022 (564h7m31.05761613s) [upload=496461]: Next returned nil segment Sep 22 02:15:55.704846542 (564h7m31.05763165s) [upload=496461]: Waiting for error group managing segments... Sep 22 02:15:55.712709156 (564h7m31.065494285s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.713081926 (564h7m31.065867054s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.716351481 (564h7m31.06913661s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717069920 (564h7m31.069855048s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717303427 (564h7m31.070088556s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725190529 (564h7m31.077975657s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725273014 (564h7m31.078058142s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732220611 (564h7m31.085005749s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732337590 (564h7m31.085122719s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.734847580 (564h7m31.087632718s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.738226892 (564h7m31.09101203s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:16:55.564886089 (564h8m30.917671218s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1AowWwVvxzmnW5qwbqbMBVbvNoigffjAN51LQp8e6XA9CZkeXL, address:145.79.227.12:4790): protocol: rpc: dial tcp 145.79.227.12:4790: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564944549 (564h8m30.917729677s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564994583 (564h8m30.917779711s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565058042 (564h8m30.91784317s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1zywsXJnw5gfpUu7mC8p7UE7ev5HexPu5nSpUH1x2pzar99T97, address:145.79.227.8:4349): protocol: rpc: dial tcp 145.79.227.8:4349: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565089361 (564h8m30.917874489s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:145.79.227.14:4847): protocol: rpc: dial tcp 145.79.227.14:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565111382 (564h8m30.91789651s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12Pxos2xcjHwcYyQnXeRL3PL72G9pXaAnBs8XnkVFkxegDRPrqp, address:145.79.227.2:4127): protocol: rpc: dial tcp 145.79.227.2:4127: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565129266 (564h8m30.917914394s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12gs9LRE3oUp1jupCw5uDvRtkNcahbq2d4VTYQsdKr6atk6vGJx, address:145.79.227.19:4095): protocol: rpc: dial tcp 145.79.227.19:4095: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565148231 (564h8m30.917933359s) [seg_pos={0 0}, upload=496461]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:16:55.580701268 (564h8m30.933486366s) [upload=496461]: Done uploading segments... Sep 22 02:16:55.580714373 (564h8m30.93349947s) [upload=496461]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 441.822241ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [...........................XXX....XXX.XXXXX..X.XX.XX.XX.....] duration: 409.404851ms FAIL (2025-09-22 02:30:26 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:26:47.079589262 (564h18m22.43237436s) [upload=496619]: Starting upload Sep 22 02:26:47.079635679 (564h18m22.432420787s) [upload=496619]: Done starting upload Sep 22 02:26:47.079683168 (564h18m22.432468266s) [upload=496619]: Uploading segments... Sep 22 02:26:47.079847085 (564h18m22.432632184s) [upload=496619]: Got next segment. Inline: false Sep 22 02:26:47.079880538 (564h18m22.432665637s) [upload=496619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:26:47.079884426 (564h18m22.432669524s) [upload=496619]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:26:47.098941027 (564h18m22.451726136s) [seg_pos={0 0}, upload=496619]: Begin upload segment... Sep 22 02:26:47.098967026 (564h18m22.451752124s) [seg_pos={0 0}, upload=496619]: Done begin upload segment. Sep 22 02:26:47.098988436 (564h18m22.451773544s) [noise=true , address=45.91.116.13:4153 , piece_id=KEVDRJJG3U53XZ42Y2N6JHAAWCI55B2LOTVXLTOLFRA7KLSIH6OA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099011058 (564h18m22.451796166s) [noise=true , address=145.79.227.2:4114 , piece_id=RIGKONJTORRHSKCZYTY5RHRKRWZG3ZPJLNP6MNFJENKINXUA3V7A, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099264234 (564h18m22.452049342s) [noise=true , address=45.91.116.9:4261 , piece_id=T36MVEHIYIU6GT23PS635SUF2K6OOU7JCSVKJ4TT6L43JEJV5CQA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099278360 (564h18m22.452063468s) [noise=true , address=145.79.227.19:4078 , piece_id=RDISZTZGXWT6C63QJ5A6ISV6YLTWMGFK43AZ37UN6TH7FB6IADUA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099503983 (564h18m22.452289081s) [noise=true , address=77.90.31.13:4626 , piece_id=NFHSBU7BRIFCYUOSOMPTJBUCJ5NBDHTRVJOET4W5IATUT6TMOYLA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099518230 (564h18m22.452303338s) [noise=true , address=145.79.227.6:4259 , piece_id=BBTEHIIGU2C7CTREQ7LF6GAIP66LMPQWS3CNHP5STLNOHSHRSALA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099716462 (564h18m22.452501561s) [noise=true , address=145.79.227.13:4804 , piece_id=TBHQSYEJUGV2XWHJIIPRWAYCLCMNKE7RKTPQY5PHDKNQJ5E5DA4A, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099730319 (564h18m22.452515417s) [noise=true , address=77.90.31.17:4771 , piece_id=MKFA2NQ6VFYBSYCYOQH7ILEZPYJ22CF3QGLQ5IMJYAPUXCAXQHUA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099964939 (564h18m22.452750047s) [noise=true , address=77.90.31.21:4878 , piece_id=64AOV3HKKUU6PCGD4SNDR3FVSGMTKKAMIV4JONEB5NIHQA2YJLIQ, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099968655 (564h18m22.452753754s) [noise=true , address=77.90.31.17:4770 , piece_id=RFZW4THLK3KTDNFJ25M3MXF2F3LD73APWI5EMZKBKIMQ72W6M65A, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100179582 (564h18m22.452964679s) [noise=true , address=45.91.116.8:4047 , piece_id=J4L2SJIV6VWR4O2VOH3PGNBPJIJG54QWRFKJAY72IWIL2UM3VBSA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100184851 (564h18m22.452969949s) [noise=true , address=77.90.31.14:4663 , piece_id=BHXXI4OQDYIR2E7PDGLEUYTLX5LRICO7THFGBS3BE4BCOWOX5NCA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100410064 (564h18m22.453195172s) [noise=true , address=77.90.31.15:4699 , piece_id=C25UYB5QNC7VOEDFN5TS26L424PTH7MQTESIQ4E24NLLHM35AABQ, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100477561 (564h18m22.453262669s) [noise=true , address=45.91.116.5:4297 , piece_id=YDYN4PVNPURJ6AHCQFVBTKVOLIZOIPKQYFJZ6GE2WR5TOEAW3IVQ, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100637391 (564h18m22.453422499s) [noise=true , address=45.91.116.4:4937 , piece_id=PL2JXVI6Q5A46PAIUGMJXRFQE4HYOPISL7KEE7CATBNLFABD75LQ, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100854929 (564h18m22.453640037s) [noise=true , address=45.91.116.6:4973 , piece_id=2KURM2DOGKAN3EDAZXIJDOPC6ACAE6GH7ORMQ45CV4TDN35DWE3Q, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.179812078 (564h18m22.532597186s) [upload=496619]: Next returned nil segment Sep 22 02:26:47.179824220 (564h18m22.532609328s) [upload=496619]: Waiting for error group managing segments... Sep 22 02:26:47.189480151 (564h18m22.542265249s) [noise=true , address=45.91.116.5:4297 , piece_id=YDYN4PVNPURJ6AHCQFVBTKVOLIZOIPKQYFJZ6GE2WR5TOEAW3IVQ, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.189604965 (564h18m22.542390064s) [noise=true , address=77.90.31.21:4878 , piece_id=64AOV3HKKUU6PCGD4SNDR3FVSGMTKKAMIV4JONEB5NIHQA2YJLIQ, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.189887455 (564h18m22.542672553s) [noise=true , address=45.91.116.8:4047 , piece_id=J4L2SJIV6VWR4O2VOH3PGNBPJIJG54QWRFKJAY72IWIL2UM3VBSA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.190226310 (564h18m22.543011409s) [noise=true , address=77.90.31.17:4770 , piece_id=RFZW4THLK3KTDNFJ25M3MXF2F3LD73APWI5EMZKBKIMQ72W6M65A, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.190460461 (564h18m22.543245559s) [noise=true , address=45.91.116.6:4973 , piece_id=2KURM2DOGKAN3EDAZXIJDOPC6ACAE6GH7ORMQ45CV4TDN35DWE3Q, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.191343909 (564h18m22.544129007s) [noise=true , address=45.91.116.9:4261 , piece_id=T36MVEHIYIU6GT23PS635SUF2K6OOU7JCSVKJ4TT6L43JEJV5CQA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.191584671 (564h18m22.544369779s) [noise=true , address=77.90.31.13:4626 , piece_id=NFHSBU7BRIFCYUOSOMPTJBUCJ5NBDHTRVJOET4W5IATUT6TMOYLA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.191710486 (564h18m22.544495584s) [noise=true , address=45.91.116.4:4937 , piece_id=PL2JXVI6Q5A46PAIUGMJXRFQE4HYOPISL7KEE7CATBNLFABD75LQ, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.192393257 (564h18m22.545178366s) [noise=true , address=77.90.31.17:4771 , piece_id=MKFA2NQ6VFYBSYCYOQH7ILEZPYJ22CF3QGLQ5IMJYAPUXCAXQHUA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.192411743 (564h18m22.54519684s) [noise=true , address=77.90.31.14:4663 , piece_id=BHXXI4OQDYIR2E7PDGLEUYTLX5LRICO7THFGBS3BE4BCOWOX5NCA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.192846608 (564h18m22.545631706s) [noise=true , address=45.91.116.13:4153 , piece_id=KEVDRJJG3U53XZ42Y2N6JHAAWCI55B2LOTVXLTOLFRA7KLSIH6OA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.194370378 (564h18m22.547155486s) [noise=true , address=77.90.31.15:4699 , piece_id=C25UYB5QNC7VOEDFN5TS26L424PTH7MQTESIQ4E24NLLHM35AABQ, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:27:17.067340274 (564h18m52.420125392s) [noise=true , address=145.79.227.2:4114 , piece_id=RIGKONJTORRHSKCZYTY5RHRKRWZG3ZPJLNP6MNFJENKINXUA3V7A, seg_pos={0 0}, upload=496619]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: rpc: dial tcp 145.79.227.2:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:27:17.067382292 (564h18m52.420167421s) [noise=true , address=145.79.227.19:4078 , piece_id=RDISZTZGXWT6C63QJ5A6ISV6YLTWMGFK43AZ37UN6TH7FB6IADUA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:27:17.067415285 (564h18m52.420200412s) [noise=true , address=145.79.227.13:4804 , piece_id=TBHQSYEJUGV2XWHJIIPRWAYCLCMNKE7RKTPQY5PHDKNQJ5E5DA4A, seg_pos={0 0}, upload=496619]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:27:17.067426626 (564h18m52.420211754s) [noise=true , address=145.79.227.6:4259 , piece_id=BBTEHIIGU2C7CTREQ7LF6GAIP66LMPQWS3CNHP5STLNOHSHRSALA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: rpc: dial tcp 145.79.227.6:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:27:17.067470808 (564h18m52.420255937s) [seg_pos={0 0}, upload=496619]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:27:17.079419761 (564h18m52.43220488s) [upload=496619]: Done uploading segments... Sep 22 02:27:17.079433988 (564h18m52.432219117s) [upload=496619]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 409.447921ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...............................X.X.........X.......X...X....] duration: 433.043738ms FAIL (2025-09-22 02:30:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 433.057953ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [...............................X.XXXXXX..X..................] duration: 383.124691ms FAIL (2025-09-22 02:30:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== m30.954857157s) [seg_pos={0 0}, upload=496461]: Begin upload segment... Sep 22 02:15:55.602116061 (564h7m30.954901189s) [seg_pos={0 0}, upload=496461]: Done begin upload segment. Sep 22 02:15:55.602137471 (564h7m30.954922599s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602154613 (564h7m30.954939742s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602716609 (564h7m30.955501736s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602736295 (564h7m30.955521423s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603134873 (564h7m30.955920001s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603640462 (564h7m30.95642559s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604036745 (564h7m30.956821874s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604397323 (564h7m30.95718245s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604828131 (564h7m30.957613268s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605250674 (564h7m30.958035802s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605663548 (564h7m30.958448676s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606022221 (564h7m30.958807349s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606387297 (564h7m30.959172425s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606869060 (564h7m30.959654198s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607319626 (564h7m30.960104755s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607717662 (564h7m30.960502801s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608211139 (564h7m30.960996268s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608652857 (564h7m30.961437986s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.704831022 (564h7m31.05761613s) [upload=496461]: Next returned nil segment Sep 22 02:15:55.704846542 (564h7m31.05763165s) [upload=496461]: Waiting for error group managing segments... Sep 22 02:15:55.712709156 (564h7m31.065494285s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.713081926 (564h7m31.065867054s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.716351481 (564h7m31.06913661s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717069920 (564h7m31.069855048s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717303427 (564h7m31.070088556s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725190529 (564h7m31.077975657s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725273014 (564h7m31.078058142s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732220611 (564h7m31.085005749s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732337590 (564h7m31.085122719s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.734847580 (564h7m31.087632718s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.738226892 (564h7m31.09101203s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:16:55.564886089 (564h8m30.917671218s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1AowWwVvxzmnW5qwbqbMBVbvNoigffjAN51LQp8e6XA9CZkeXL, address:145.79.227.12:4790): protocol: rpc: dial tcp 145.79.227.12:4790: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564944549 (564h8m30.917729677s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564994583 (564h8m30.917779711s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565058042 (564h8m30.91784317s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1zywsXJnw5gfpUu7mC8p7UE7ev5HexPu5nSpUH1x2pzar99T97, address:145.79.227.8:4349): protocol: rpc: dial tcp 145.79.227.8:4349: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565089361 (564h8m30.917874489s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:145.79.227.14:4847): protocol: rpc: dial tcp 145.79.227.14:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565111382 (564h8m30.91789651s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12Pxos2xcjHwcYyQnXeRL3PL72G9pXaAnBs8XnkVFkxegDRPrqp, address:145.79.227.2:4127): protocol: rpc: dial tcp 145.79.227.2:4127: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565129266 (564h8m30.917914394s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12gs9LRE3oUp1jupCw5uDvRtkNcahbq2d4VTYQsdKr6atk6vGJx, address:145.79.227.19:4095): protocol: rpc: dial tcp 145.79.227.19:4095: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565148231 (564h8m30.917933359s) [seg_pos={0 0}, upload=496461]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:16:55.580701268 (564h8m30.933486366s) [upload=496461]: Done uploading segments... Sep 22 02:16:55.580714373 (564h8m30.93349947s) [upload=496461]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 383.149558ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [............................XXX....XXX.XXXXX..X.XX.XX.XX....] duration: 461.110206ms FAIL (2025-09-22 02:30:06 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:26:47.079589262 (564h18m22.43237436s) [upload=496619]: Starting upload Sep 22 02:26:47.079635679 (564h18m22.432420787s) [upload=496619]: Done starting upload Sep 22 02:26:47.079683168 (564h18m22.432468266s) [upload=496619]: Uploading segments... Sep 22 02:26:47.079847085 (564h18m22.432632184s) [upload=496619]: Got next segment. Inline: false Sep 22 02:26:47.079880538 (564h18m22.432665637s) [upload=496619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:26:47.079884426 (564h18m22.432669524s) [upload=496619]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:26:47.098941027 (564h18m22.451726136s) [seg_pos={0 0}, upload=496619]: Begin upload segment... Sep 22 02:26:47.098967026 (564h18m22.451752124s) [seg_pos={0 0}, upload=496619]: Done begin upload segment. Sep 22 02:26:47.098988436 (564h18m22.451773544s) [noise=true , address=45.91.116.13:4153 , piece_id=KEVDRJJG3U53XZ42Y2N6JHAAWCI55B2LOTVXLTOLFRA7KLSIH6OA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099011058 (564h18m22.451796166s) [noise=true , address=145.79.227.2:4114 , piece_id=RIGKONJTORRHSKCZYTY5RHRKRWZG3ZPJLNP6MNFJENKINXUA3V7A, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099264234 (564h18m22.452049342s) [noise=true , address=45.91.116.9:4261 , piece_id=T36MVEHIYIU6GT23PS635SUF2K6OOU7JCSVKJ4TT6L43JEJV5CQA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099278360 (564h18m22.452063468s) [noise=true , address=145.79.227.19:4078 , piece_id=RDISZTZGXWT6C63QJ5A6ISV6YLTWMGFK43AZ37UN6TH7FB6IADUA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099503983 (564h18m22.452289081s) [noise=true , address=77.90.31.13:4626 , piece_id=NFHSBU7BRIFCYUOSOMPTJBUCJ5NBDHTRVJOET4W5IATUT6TMOYLA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099518230 (564h18m22.452303338s) [noise=true , address=145.79.227.6:4259 , piece_id=BBTEHIIGU2C7CTREQ7LF6GAIP66LMPQWS3CNHP5STLNOHSHRSALA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099716462 (564h18m22.452501561s) [noise=true , address=145.79.227.13:4804 , piece_id=TBHQSYEJUGV2XWHJIIPRWAYCLCMNKE7RKTPQY5PHDKNQJ5E5DA4A, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099730319 (564h18m22.452515417s) [noise=true , address=77.90.31.17:4771 , piece_id=MKFA2NQ6VFYBSYCYOQH7ILEZPYJ22CF3QGLQ5IMJYAPUXCAXQHUA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099964939 (564h18m22.452750047s) [noise=true , address=77.90.31.21:4878 , piece_id=64AOV3HKKUU6PCGD4SNDR3FVSGMTKKAMIV4JONEB5NIHQA2YJLIQ, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099968655 (564h18m22.452753754s) [noise=true , address=77.90.31.17:4770 , piece_id=RFZW4THLK3KTDNFJ25M3MXF2F3LD73APWI5EMZKBKIMQ72W6M65A, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100179582 (564h18m22.452964679s) [noise=true , address=45.91.116.8:4047 , piece_id=J4L2SJIV6VWR4O2VOH3PGNBPJIJG54QWRFKJAY72IWIL2UM3VBSA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100184851 (564h18m22.452969949s) [noise=true , address=77.90.31.14:4663 , piece_id=BHXXI4OQDYIR2E7PDGLEUYTLX5LRICO7THFGBS3BE4BCOWOX5NCA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100410064 (564h18m22.453195172s) [noise=true , address=77.90.31.15:4699 , piece_id=C25UYB5QNC7VOEDFN5TS26L424PTH7MQTESIQ4E24NLLHM35AABQ, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100477561 (564h18m22.453262669s) [noise=true , address=45.91.116.5:4297 , piece_id=YDYN4PVNPURJ6AHCQFVBTKVOLIZOIPKQYFJZ6GE2WR5TOEAW3IVQ, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100637391 (564h18m22.453422499s) [noise=true , address=45.91.116.4:4937 , piece_id=PL2JXVI6Q5A46PAIUGMJXRFQE4HYOPISL7KEE7CATBNLFABD75LQ, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100854929 (564h18m22.453640037s) [noise=true , address=45.91.116.6:4973 , piece_id=2KURM2DOGKAN3EDAZXIJDOPC6ACAE6GH7ORMQ45CV4TDN35DWE3Q, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.179812078 (564h18m22.532597186s) [upload=496619]: Next returned nil segment Sep 22 02:26:47.179824220 (564h18m22.532609328s) [upload=496619]: Waiting for error group managing segments... Sep 22 02:26:47.189480151 (564h18m22.542265249s) [noise=true , address=45.91.116.5:4297 , piece_id=YDYN4PVNPURJ6AHCQFVBTKVOLIZOIPKQYFJZ6GE2WR5TOEAW3IVQ, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.189604965 (564h18m22.542390064s) [noise=true , address=77.90.31.21:4878 , piece_id=64AOV3HKKUU6PCGD4SNDR3FVSGMTKKAMIV4JONEB5NIHQA2YJLIQ, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.189887455 (564h18m22.542672553s) [noise=true , address=45.91.116.8:4047 , piece_id=J4L2SJIV6VWR4O2VOH3PGNBPJIJG54QWRFKJAY72IWIL2UM3VBSA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.190226310 (564h18m22.543011409s) [noise=true , address=77.90.31.17:4770 , piece_id=RFZW4THLK3KTDNFJ25M3MXF2F3LD73APWI5EMZKBKIMQ72W6M65A, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.190460461 (564h18m22.543245559s) [noise=true , address=45.91.116.6:4973 , piece_id=2KURM2DOGKAN3EDAZXIJDOPC6ACAE6GH7ORMQ45CV4TDN35DWE3Q, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.191343909 (564h18m22.544129007s) [noise=true , address=45.91.116.9:4261 , piece_id=T36MVEHIYIU6GT23PS635SUF2K6OOU7JCSVKJ4TT6L43JEJV5CQA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.191584671 (564h18m22.544369779s) [noise=true , address=77.90.31.13:4626 , piece_id=NFHSBU7BRIFCYUOSOMPTJBUCJ5NBDHTRVJOET4W5IATUT6TMOYLA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.191710486 (564h18m22.544495584s) [noise=true , address=45.91.116.4:4937 , piece_id=PL2JXVI6Q5A46PAIUGMJXRFQE4HYOPISL7KEE7CATBNLFABD75LQ, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.192393257 (564h18m22.545178366s) [noise=true , address=77.90.31.17:4771 , piece_id=MKFA2NQ6VFYBSYCYOQH7ILEZPYJ22CF3QGLQ5IMJYAPUXCAXQHUA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.192411743 (564h18m22.54519684s) [noise=true , address=77.90.31.14:4663 , piece_id=BHXXI4OQDYIR2E7PDGLEUYTLX5LRICO7THFGBS3BE4BCOWOX5NCA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.192846608 (564h18m22.545631706s) [noise=true , address=45.91.116.13:4153 , piece_id=KEVDRJJG3U53XZ42Y2N6JHAAWCI55B2LOTVXLTOLFRA7KLSIH6OA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.194370378 (564h18m22.547155486s) [noise=true , address=77.90.31.15:4699 , piece_id=C25UYB5QNC7VOEDFN5TS26L424PTH7MQTESIQ4E24NLLHM35AABQ, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:27:17.067340274 (564h18m52.420125392s) [noise=true , address=145.79.227.2:4114 , piece_id=RIGKONJTORRHSKCZYTY5RHRKRWZG3ZPJLNP6MNFJENKINXUA3V7A, seg_pos={0 0}, upload=496619]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: rpc: dial tcp 145.79.227.2:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:27:17.067382292 (564h18m52.420167421s) [noise=true , address=145.79.227.19:4078 , piece_id=RDISZTZGXWT6C63QJ5A6ISV6YLTWMGFK43AZ37UN6TH7FB6IADUA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:27:17.067415285 (564h18m52.420200412s) [noise=true , address=145.79.227.13:4804 , piece_id=TBHQSYEJUGV2XWHJIIPRWAYCLCMNKE7RKTPQY5PHDKNQJ5E5DA4A, seg_pos={0 0}, upload=496619]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:27:17.067426626 (564h18m52.420211754s) [noise=true , address=145.79.227.6:4259 , piece_id=BBTEHIIGU2C7CTREQ7LF6GAIP66LMPQWS3CNHP5STLNOHSHRSALA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: rpc: dial tcp 145.79.227.6:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:27:17.067470808 (564h18m52.420255937s) [seg_pos={0 0}, upload=496619]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:27:17.079419761 (564h18m52.43220488s) [upload=496619]: Done uploading segments... Sep 22 02:27:17.079433988 (564h18m52.432219117s) [upload=496619]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 461.135164ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [................................X.XXXXXX..X.................] duration: 557.387647ms FAIL (2025-09-22 02:29:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== m30.954857157s) [seg_pos={0 0}, upload=496461]: Begin upload segment... Sep 22 02:15:55.602116061 (564h7m30.954901189s) [seg_pos={0 0}, upload=496461]: Done begin upload segment. Sep 22 02:15:55.602137471 (564h7m30.954922599s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602154613 (564h7m30.954939742s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602716609 (564h7m30.955501736s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602736295 (564h7m30.955521423s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603134873 (564h7m30.955920001s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603640462 (564h7m30.95642559s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604036745 (564h7m30.956821874s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604397323 (564h7m30.95718245s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604828131 (564h7m30.957613268s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605250674 (564h7m30.958035802s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605663548 (564h7m30.958448676s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606022221 (564h7m30.958807349s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606387297 (564h7m30.959172425s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606869060 (564h7m30.959654198s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607319626 (564h7m30.960104755s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607717662 (564h7m30.960502801s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608211139 (564h7m30.960996268s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608652857 (564h7m30.961437986s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.704831022 (564h7m31.05761613s) [upload=496461]: Next returned nil segment Sep 22 02:15:55.704846542 (564h7m31.05763165s) [upload=496461]: Waiting for error group managing segments... Sep 22 02:15:55.712709156 (564h7m31.065494285s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.713081926 (564h7m31.065867054s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.716351481 (564h7m31.06913661s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717069920 (564h7m31.069855048s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717303427 (564h7m31.070088556s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725190529 (564h7m31.077975657s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725273014 (564h7m31.078058142s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732220611 (564h7m31.085005749s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732337590 (564h7m31.085122719s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.734847580 (564h7m31.087632718s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.738226892 (564h7m31.09101203s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:16:55.564886089 (564h8m30.917671218s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1AowWwVvxzmnW5qwbqbMBVbvNoigffjAN51LQp8e6XA9CZkeXL, address:145.79.227.12:4790): protocol: rpc: dial tcp 145.79.227.12:4790: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564944549 (564h8m30.917729677s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564994583 (564h8m30.917779711s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565058042 (564h8m30.91784317s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1zywsXJnw5gfpUu7mC8p7UE7ev5HexPu5nSpUH1x2pzar99T97, address:145.79.227.8:4349): protocol: rpc: dial tcp 145.79.227.8:4349: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565089361 (564h8m30.917874489s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:145.79.227.14:4847): protocol: rpc: dial tcp 145.79.227.14:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565111382 (564h8m30.91789651s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12Pxos2xcjHwcYyQnXeRL3PL72G9pXaAnBs8XnkVFkxegDRPrqp, address:145.79.227.2:4127): protocol: rpc: dial tcp 145.79.227.2:4127: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565129266 (564h8m30.917914394s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12gs9LRE3oUp1jupCw5uDvRtkNcahbq2d4VTYQsdKr6atk6vGJx, address:145.79.227.19:4095): protocol: rpc: dial tcp 145.79.227.19:4095: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565148231 (564h8m30.917933359s) [seg_pos={0 0}, upload=496461]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:16:55.580701268 (564h8m30.933486366s) [upload=496461]: Done uploading segments... Sep 22 02:16:55.580714373 (564h8m30.93349947s) [upload=496461]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 557.411962ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [................................X.X.........X.......X...X...] duration: 459.614819ms FAIL (2025-09-22 02:29:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 459.634475ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [.............................XXX....XXX.XXXXX..X.XX.XX.XX...] duration: 555.3038ms FAIL (2025-09-22 02:29:42 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:26:47.079589262 (564h18m22.43237436s) [upload=496619]: Starting upload Sep 22 02:26:47.079635679 (564h18m22.432420787s) [upload=496619]: Done starting upload Sep 22 02:26:47.079683168 (564h18m22.432468266s) [upload=496619]: Uploading segments... Sep 22 02:26:47.079847085 (564h18m22.432632184s) [upload=496619]: Got next segment. Inline: false Sep 22 02:26:47.079880538 (564h18m22.432665637s) [upload=496619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:26:47.079884426 (564h18m22.432669524s) [upload=496619]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:26:47.098941027 (564h18m22.451726136s) [seg_pos={0 0}, upload=496619]: Begin upload segment... Sep 22 02:26:47.098967026 (564h18m22.451752124s) [seg_pos={0 0}, upload=496619]: Done begin upload segment. Sep 22 02:26:47.098988436 (564h18m22.451773544s) [noise=true , address=45.91.116.13:4153 , piece_id=KEVDRJJG3U53XZ42Y2N6JHAAWCI55B2LOTVXLTOLFRA7KLSIH6OA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099011058 (564h18m22.451796166s) [noise=true , address=145.79.227.2:4114 , piece_id=RIGKONJTORRHSKCZYTY5RHRKRWZG3ZPJLNP6MNFJENKINXUA3V7A, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099264234 (564h18m22.452049342s) [noise=true , address=45.91.116.9:4261 , piece_id=T36MVEHIYIU6GT23PS635SUF2K6OOU7JCSVKJ4TT6L43JEJV5CQA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099278360 (564h18m22.452063468s) [noise=true , address=145.79.227.19:4078 , piece_id=RDISZTZGXWT6C63QJ5A6ISV6YLTWMGFK43AZ37UN6TH7FB6IADUA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099503983 (564h18m22.452289081s) [noise=true , address=77.90.31.13:4626 , piece_id=NFHSBU7BRIFCYUOSOMPTJBUCJ5NBDHTRVJOET4W5IATUT6TMOYLA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099518230 (564h18m22.452303338s) [noise=true , address=145.79.227.6:4259 , piece_id=BBTEHIIGU2C7CTREQ7LF6GAIP66LMPQWS3CNHP5STLNOHSHRSALA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099716462 (564h18m22.452501561s) [noise=true , address=145.79.227.13:4804 , piece_id=TBHQSYEJUGV2XWHJIIPRWAYCLCMNKE7RKTPQY5PHDKNQJ5E5DA4A, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099730319 (564h18m22.452515417s) [noise=true , address=77.90.31.17:4771 , piece_id=MKFA2NQ6VFYBSYCYOQH7ILEZPYJ22CF3QGLQ5IMJYAPUXCAXQHUA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099964939 (564h18m22.452750047s) [noise=true , address=77.90.31.21:4878 , piece_id=64AOV3HKKUU6PCGD4SNDR3FVSGMTKKAMIV4JONEB5NIHQA2YJLIQ, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099968655 (564h18m22.452753754s) [noise=true , address=77.90.31.17:4770 , piece_id=RFZW4THLK3KTDNFJ25M3MXF2F3LD73APWI5EMZKBKIMQ72W6M65A, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100179582 (564h18m22.452964679s) [noise=true , address=45.91.116.8:4047 , piece_id=J4L2SJIV6VWR4O2VOH3PGNBPJIJG54QWRFKJAY72IWIL2UM3VBSA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100184851 (564h18m22.452969949s) [noise=true , address=77.90.31.14:4663 , piece_id=BHXXI4OQDYIR2E7PDGLEUYTLX5LRICO7THFGBS3BE4BCOWOX5NCA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100410064 (564h18m22.453195172s) [noise=true , address=77.90.31.15:4699 , piece_id=C25UYB5QNC7VOEDFN5TS26L424PTH7MQTESIQ4E24NLLHM35AABQ, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100477561 (564h18m22.453262669s) [noise=true , address=45.91.116.5:4297 , piece_id=YDYN4PVNPURJ6AHCQFVBTKVOLIZOIPKQYFJZ6GE2WR5TOEAW3IVQ, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100637391 (564h18m22.453422499s) [noise=true , address=45.91.116.4:4937 , piece_id=PL2JXVI6Q5A46PAIUGMJXRFQE4HYOPISL7KEE7CATBNLFABD75LQ, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100854929 (564h18m22.453640037s) [noise=true , address=45.91.116.6:4973 , piece_id=2KURM2DOGKAN3EDAZXIJDOPC6ACAE6GH7ORMQ45CV4TDN35DWE3Q, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.179812078 (564h18m22.532597186s) [upload=496619]: Next returned nil segment Sep 22 02:26:47.179824220 (564h18m22.532609328s) [upload=496619]: Waiting for error group managing segments... Sep 22 02:26:47.189480151 (564h18m22.542265249s) [noise=true , address=45.91.116.5:4297 , piece_id=YDYN4PVNPURJ6AHCQFVBTKVOLIZOIPKQYFJZ6GE2WR5TOEAW3IVQ, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.189604965 (564h18m22.542390064s) [noise=true , address=77.90.31.21:4878 , piece_id=64AOV3HKKUU6PCGD4SNDR3FVSGMTKKAMIV4JONEB5NIHQA2YJLIQ, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.189887455 (564h18m22.542672553s) [noise=true , address=45.91.116.8:4047 , piece_id=J4L2SJIV6VWR4O2VOH3PGNBPJIJG54QWRFKJAY72IWIL2UM3VBSA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.190226310 (564h18m22.543011409s) [noise=true , address=77.90.31.17:4770 , piece_id=RFZW4THLK3KTDNFJ25M3MXF2F3LD73APWI5EMZKBKIMQ72W6M65A, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.190460461 (564h18m22.543245559s) [noise=true , address=45.91.116.6:4973 , piece_id=2KURM2DOGKAN3EDAZXIJDOPC6ACAE6GH7ORMQ45CV4TDN35DWE3Q, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.191343909 (564h18m22.544129007s) [noise=true , address=45.91.116.9:4261 , piece_id=T36MVEHIYIU6GT23PS635SUF2K6OOU7JCSVKJ4TT6L43JEJV5CQA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.191584671 (564h18m22.544369779s) [noise=true , address=77.90.31.13:4626 , piece_id=NFHSBU7BRIFCYUOSOMPTJBUCJ5NBDHTRVJOET4W5IATUT6TMOYLA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.191710486 (564h18m22.544495584s) [noise=true , address=45.91.116.4:4937 , piece_id=PL2JXVI6Q5A46PAIUGMJXRFQE4HYOPISL7KEE7CATBNLFABD75LQ, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.192393257 (564h18m22.545178366s) [noise=true , address=77.90.31.17:4771 , piece_id=MKFA2NQ6VFYBSYCYOQH7ILEZPYJ22CF3QGLQ5IMJYAPUXCAXQHUA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.192411743 (564h18m22.54519684s) [noise=true , address=77.90.31.14:4663 , piece_id=BHXXI4OQDYIR2E7PDGLEUYTLX5LRICO7THFGBS3BE4BCOWOX5NCA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.192846608 (564h18m22.545631706s) [noise=true , address=45.91.116.13:4153 , piece_id=KEVDRJJG3U53XZ42Y2N6JHAAWCI55B2LOTVXLTOLFRA7KLSIH6OA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.194370378 (564h18m22.547155486s) [noise=true , address=77.90.31.15:4699 , piece_id=C25UYB5QNC7VOEDFN5TS26L424PTH7MQTESIQ4E24NLLHM35AABQ, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:27:17.067340274 (564h18m52.420125392s) [noise=true , address=145.79.227.2:4114 , piece_id=RIGKONJTORRHSKCZYTY5RHRKRWZG3ZPJLNP6MNFJENKINXUA3V7A, seg_pos={0 0}, upload=496619]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: rpc: dial tcp 145.79.227.2:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:27:17.067382292 (564h18m52.420167421s) [noise=true , address=145.79.227.19:4078 , piece_id=RDISZTZGXWT6C63QJ5A6ISV6YLTWMGFK43AZ37UN6TH7FB6IADUA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:27:17.067415285 (564h18m52.420200412s) [noise=true , address=145.79.227.13:4804 , piece_id=TBHQSYEJUGV2XWHJIIPRWAYCLCMNKE7RKTPQY5PHDKNQJ5E5DA4A, seg_pos={0 0}, upload=496619]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:27:17.067426626 (564h18m52.420211754s) [noise=true , address=145.79.227.6:4259 , piece_id=BBTEHIIGU2C7CTREQ7LF6GAIP66LMPQWS3CNHP5STLNOHSHRSALA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: rpc: dial tcp 145.79.227.6:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:27:17.067470808 (564h18m52.420255937s) [seg_pos={0 0}, upload=496619]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:27:17.079419761 (564h18m52.43220488s) [upload=496619]: Done uploading segments... Sep 22 02:27:17.079433988 (564h18m52.432219117s) [upload=496619]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 555.330089ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [.................................X.XXXXXX..X................] duration: 505.203561ms FAIL (2025-09-22 02:29:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== m30.954857157s) [seg_pos={0 0}, upload=496461]: Begin upload segment... Sep 22 02:15:55.602116061 (564h7m30.954901189s) [seg_pos={0 0}, upload=496461]: Done begin upload segment. Sep 22 02:15:55.602137471 (564h7m30.954922599s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602154613 (564h7m30.954939742s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602716609 (564h7m30.955501736s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602736295 (564h7m30.955521423s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603134873 (564h7m30.955920001s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603640462 (564h7m30.95642559s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604036745 (564h7m30.956821874s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604397323 (564h7m30.95718245s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604828131 (564h7m30.957613268s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605250674 (564h7m30.958035802s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605663548 (564h7m30.958448676s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606022221 (564h7m30.958807349s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606387297 (564h7m30.959172425s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606869060 (564h7m30.959654198s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607319626 (564h7m30.960104755s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607717662 (564h7m30.960502801s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608211139 (564h7m30.960996268s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608652857 (564h7m30.961437986s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.704831022 (564h7m31.05761613s) [upload=496461]: Next returned nil segment Sep 22 02:15:55.704846542 (564h7m31.05763165s) [upload=496461]: Waiting for error group managing segments... Sep 22 02:15:55.712709156 (564h7m31.065494285s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.713081926 (564h7m31.065867054s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.716351481 (564h7m31.06913661s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717069920 (564h7m31.069855048s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717303427 (564h7m31.070088556s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725190529 (564h7m31.077975657s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725273014 (564h7m31.078058142s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732220611 (564h7m31.085005749s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732337590 (564h7m31.085122719s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.734847580 (564h7m31.087632718s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.738226892 (564h7m31.09101203s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:16:55.564886089 (564h8m30.917671218s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1AowWwVvxzmnW5qwbqbMBVbvNoigffjAN51LQp8e6XA9CZkeXL, address:145.79.227.12:4790): protocol: rpc: dial tcp 145.79.227.12:4790: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564944549 (564h8m30.917729677s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564994583 (564h8m30.917779711s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565058042 (564h8m30.91784317s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1zywsXJnw5gfpUu7mC8p7UE7ev5HexPu5nSpUH1x2pzar99T97, address:145.79.227.8:4349): protocol: rpc: dial tcp 145.79.227.8:4349: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565089361 (564h8m30.917874489s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:145.79.227.14:4847): protocol: rpc: dial tcp 145.79.227.14:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565111382 (564h8m30.91789651s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12Pxos2xcjHwcYyQnXeRL3PL72G9pXaAnBs8XnkVFkxegDRPrqp, address:145.79.227.2:4127): protocol: rpc: dial tcp 145.79.227.2:4127: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565129266 (564h8m30.917914394s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12gs9LRE3oUp1jupCw5uDvRtkNcahbq2d4VTYQsdKr6atk6vGJx, address:145.79.227.19:4095): protocol: rpc: dial tcp 145.79.227.19:4095: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565148231 (564h8m30.917933359s) [seg_pos={0 0}, upload=496461]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:16:55.580701268 (564h8m30.933486366s) [upload=496461]: Done uploading segments... Sep 22 02:16:55.580714373 (564h8m30.93349947s) [upload=496461]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 505.221944ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.................................X.X.........X.......X...X..] duration: 486.345442ms FAIL (2025-09-22 02:29:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 486.364969ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [..............................XXX....XXX.XXXXX..X.XX.XX.XX..] duration: 524.614725ms FAIL (2025-09-22 02:28:34 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:26:47.079589262 (564h18m22.43237436s) [upload=496619]: Starting upload Sep 22 02:26:47.079635679 (564h18m22.432420787s) [upload=496619]: Done starting upload Sep 22 02:26:47.079683168 (564h18m22.432468266s) [upload=496619]: Uploading segments... Sep 22 02:26:47.079847085 (564h18m22.432632184s) [upload=496619]: Got next segment. Inline: false Sep 22 02:26:47.079880538 (564h18m22.432665637s) [upload=496619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:26:47.079884426 (564h18m22.432669524s) [upload=496619]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:26:47.098941027 (564h18m22.451726136s) [seg_pos={0 0}, upload=496619]: Begin upload segment... Sep 22 02:26:47.098967026 (564h18m22.451752124s) [seg_pos={0 0}, upload=496619]: Done begin upload segment. Sep 22 02:26:47.098988436 (564h18m22.451773544s) [noise=true , address=45.91.116.13:4153 , piece_id=KEVDRJJG3U53XZ42Y2N6JHAAWCI55B2LOTVXLTOLFRA7KLSIH6OA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099011058 (564h18m22.451796166s) [noise=true , address=145.79.227.2:4114 , piece_id=RIGKONJTORRHSKCZYTY5RHRKRWZG3ZPJLNP6MNFJENKINXUA3V7A, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099264234 (564h18m22.452049342s) [noise=true , address=45.91.116.9:4261 , piece_id=T36MVEHIYIU6GT23PS635SUF2K6OOU7JCSVKJ4TT6L43JEJV5CQA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099278360 (564h18m22.452063468s) [noise=true , address=145.79.227.19:4078 , piece_id=RDISZTZGXWT6C63QJ5A6ISV6YLTWMGFK43AZ37UN6TH7FB6IADUA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099503983 (564h18m22.452289081s) [noise=true , address=77.90.31.13:4626 , piece_id=NFHSBU7BRIFCYUOSOMPTJBUCJ5NBDHTRVJOET4W5IATUT6TMOYLA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099518230 (564h18m22.452303338s) [noise=true , address=145.79.227.6:4259 , piece_id=BBTEHIIGU2C7CTREQ7LF6GAIP66LMPQWS3CNHP5STLNOHSHRSALA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099716462 (564h18m22.452501561s) [noise=true , address=145.79.227.13:4804 , piece_id=TBHQSYEJUGV2XWHJIIPRWAYCLCMNKE7RKTPQY5PHDKNQJ5E5DA4A, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099730319 (564h18m22.452515417s) [noise=true , address=77.90.31.17:4771 , piece_id=MKFA2NQ6VFYBSYCYOQH7ILEZPYJ22CF3QGLQ5IMJYAPUXCAXQHUA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099964939 (564h18m22.452750047s) [noise=true , address=77.90.31.21:4878 , piece_id=64AOV3HKKUU6PCGD4SNDR3FVSGMTKKAMIV4JONEB5NIHQA2YJLIQ, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099968655 (564h18m22.452753754s) [noise=true , address=77.90.31.17:4770 , piece_id=RFZW4THLK3KTDNFJ25M3MXF2F3LD73APWI5EMZKBKIMQ72W6M65A, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100179582 (564h18m22.452964679s) [noise=true , address=45.91.116.8:4047 , piece_id=J4L2SJIV6VWR4O2VOH3PGNBPJIJG54QWRFKJAY72IWIL2UM3VBSA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100184851 (564h18m22.452969949s) [noise=true , address=77.90.31.14:4663 , piece_id=BHXXI4OQDYIR2E7PDGLEUYTLX5LRICO7THFGBS3BE4BCOWOX5NCA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100410064 (564h18m22.453195172s) [noise=true , address=77.90.31.15:4699 , piece_id=C25UYB5QNC7VOEDFN5TS26L424PTH7MQTESIQ4E24NLLHM35AABQ, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100477561 (564h18m22.453262669s) [noise=true , address=45.91.116.5:4297 , piece_id=YDYN4PVNPURJ6AHCQFVBTKVOLIZOIPKQYFJZ6GE2WR5TOEAW3IVQ, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100637391 (564h18m22.453422499s) [noise=true , address=45.91.116.4:4937 , piece_id=PL2JXVI6Q5A46PAIUGMJXRFQE4HYOPISL7KEE7CATBNLFABD75LQ, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100854929 (564h18m22.453640037s) [noise=true , address=45.91.116.6:4973 , piece_id=2KURM2DOGKAN3EDAZXIJDOPC6ACAE6GH7ORMQ45CV4TDN35DWE3Q, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.179812078 (564h18m22.532597186s) [upload=496619]: Next returned nil segment Sep 22 02:26:47.179824220 (564h18m22.532609328s) [upload=496619]: Waiting for error group managing segments... Sep 22 02:26:47.189480151 (564h18m22.542265249s) [noise=true , address=45.91.116.5:4297 , piece_id=YDYN4PVNPURJ6AHCQFVBTKVOLIZOIPKQYFJZ6GE2WR5TOEAW3IVQ, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.189604965 (564h18m22.542390064s) [noise=true , address=77.90.31.21:4878 , piece_id=64AOV3HKKUU6PCGD4SNDR3FVSGMTKKAMIV4JONEB5NIHQA2YJLIQ, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.189887455 (564h18m22.542672553s) [noise=true , address=45.91.116.8:4047 , piece_id=J4L2SJIV6VWR4O2VOH3PGNBPJIJG54QWRFKJAY72IWIL2UM3VBSA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.190226310 (564h18m22.543011409s) [noise=true , address=77.90.31.17:4770 , piece_id=RFZW4THLK3KTDNFJ25M3MXF2F3LD73APWI5EMZKBKIMQ72W6M65A, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.190460461 (564h18m22.543245559s) [noise=true , address=45.91.116.6:4973 , piece_id=2KURM2DOGKAN3EDAZXIJDOPC6ACAE6GH7ORMQ45CV4TDN35DWE3Q, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.191343909 (564h18m22.544129007s) [noise=true , address=45.91.116.9:4261 , piece_id=T36MVEHIYIU6GT23PS635SUF2K6OOU7JCSVKJ4TT6L43JEJV5CQA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.191584671 (564h18m22.544369779s) [noise=true , address=77.90.31.13:4626 , piece_id=NFHSBU7BRIFCYUOSOMPTJBUCJ5NBDHTRVJOET4W5IATUT6TMOYLA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.191710486 (564h18m22.544495584s) [noise=true , address=45.91.116.4:4937 , piece_id=PL2JXVI6Q5A46PAIUGMJXRFQE4HYOPISL7KEE7CATBNLFABD75LQ, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.192393257 (564h18m22.545178366s) [noise=true , address=77.90.31.17:4771 , piece_id=MKFA2NQ6VFYBSYCYOQH7ILEZPYJ22CF3QGLQ5IMJYAPUXCAXQHUA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.192411743 (564h18m22.54519684s) [noise=true , address=77.90.31.14:4663 , piece_id=BHXXI4OQDYIR2E7PDGLEUYTLX5LRICO7THFGBS3BE4BCOWOX5NCA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.192846608 (564h18m22.545631706s) [noise=true , address=45.91.116.13:4153 , piece_id=KEVDRJJG3U53XZ42Y2N6JHAAWCI55B2LOTVXLTOLFRA7KLSIH6OA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.194370378 (564h18m22.547155486s) [noise=true , address=77.90.31.15:4699 , piece_id=C25UYB5QNC7VOEDFN5TS26L424PTH7MQTESIQ4E24NLLHM35AABQ, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:27:17.067340274 (564h18m52.420125392s) [noise=true , address=145.79.227.2:4114 , piece_id=RIGKONJTORRHSKCZYTY5RHRKRWZG3ZPJLNP6MNFJENKINXUA3V7A, seg_pos={0 0}, upload=496619]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: rpc: dial tcp 145.79.227.2:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:27:17.067382292 (564h18m52.420167421s) [noise=true , address=145.79.227.19:4078 , piece_id=RDISZTZGXWT6C63QJ5A6ISV6YLTWMGFK43AZ37UN6TH7FB6IADUA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:27:17.067415285 (564h18m52.420200412s) [noise=true , address=145.79.227.13:4804 , piece_id=TBHQSYEJUGV2XWHJIIPRWAYCLCMNKE7RKTPQY5PHDKNQJ5E5DA4A, seg_pos={0 0}, upload=496619]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:27:17.067426626 (564h18m52.420211754s) [noise=true , address=145.79.227.6:4259 , piece_id=BBTEHIIGU2C7CTREQ7LF6GAIP66LMPQWS3CNHP5STLNOHSHRSALA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: rpc: dial tcp 145.79.227.6:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:27:17.067470808 (564h18m52.420255937s) [seg_pos={0 0}, upload=496619]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:27:17.079419761 (564h18m52.43220488s) [upload=496619]: Done uploading segments... Sep 22 02:27:17.079433988 (564h18m52.432219117s) [upload=496619]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 524.639092ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [..................................X.XXXXXX..X...............] duration: 481.905472ms FAIL (2025-09-22 02:28:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== m30.954857157s) [seg_pos={0 0}, upload=496461]: Begin upload segment... Sep 22 02:15:55.602116061 (564h7m30.954901189s) [seg_pos={0 0}, upload=496461]: Done begin upload segment. Sep 22 02:15:55.602137471 (564h7m30.954922599s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602154613 (564h7m30.954939742s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602716609 (564h7m30.955501736s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602736295 (564h7m30.955521423s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603134873 (564h7m30.955920001s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603640462 (564h7m30.95642559s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604036745 (564h7m30.956821874s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604397323 (564h7m30.95718245s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604828131 (564h7m30.957613268s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605250674 (564h7m30.958035802s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605663548 (564h7m30.958448676s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606022221 (564h7m30.958807349s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606387297 (564h7m30.959172425s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606869060 (564h7m30.959654198s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607319626 (564h7m30.960104755s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607717662 (564h7m30.960502801s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608211139 (564h7m30.960996268s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608652857 (564h7m30.961437986s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.704831022 (564h7m31.05761613s) [upload=496461]: Next returned nil segment Sep 22 02:15:55.704846542 (564h7m31.05763165s) [upload=496461]: Waiting for error group managing segments... Sep 22 02:15:55.712709156 (564h7m31.065494285s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.713081926 (564h7m31.065867054s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.716351481 (564h7m31.06913661s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717069920 (564h7m31.069855048s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717303427 (564h7m31.070088556s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725190529 (564h7m31.077975657s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725273014 (564h7m31.078058142s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732220611 (564h7m31.085005749s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732337590 (564h7m31.085122719s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.734847580 (564h7m31.087632718s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.738226892 (564h7m31.09101203s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:16:55.564886089 (564h8m30.917671218s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1AowWwVvxzmnW5qwbqbMBVbvNoigffjAN51LQp8e6XA9CZkeXL, address:145.79.227.12:4790): protocol: rpc: dial tcp 145.79.227.12:4790: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564944549 (564h8m30.917729677s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564994583 (564h8m30.917779711s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565058042 (564h8m30.91784317s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1zywsXJnw5gfpUu7mC8p7UE7ev5HexPu5nSpUH1x2pzar99T97, address:145.79.227.8:4349): protocol: rpc: dial tcp 145.79.227.8:4349: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565089361 (564h8m30.917874489s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:145.79.227.14:4847): protocol: rpc: dial tcp 145.79.227.14:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565111382 (564h8m30.91789651s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12Pxos2xcjHwcYyQnXeRL3PL72G9pXaAnBs8XnkVFkxegDRPrqp, address:145.79.227.2:4127): protocol: rpc: dial tcp 145.79.227.2:4127: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565129266 (564h8m30.917914394s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12gs9LRE3oUp1jupCw5uDvRtkNcahbq2d4VTYQsdKr6atk6vGJx, address:145.79.227.19:4095): protocol: rpc: dial tcp 145.79.227.19:4095: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565148231 (564h8m30.917933359s) [seg_pos={0 0}, upload=496461]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:16:55.580701268 (564h8m30.933486366s) [upload=496461]: Done uploading segments... Sep 22 02:16:55.580714373 (564h8m30.93349947s) [upload=496461]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 481.938554ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..................................X.X.........X.......X...X.] duration: 459.76389ms FAIL (2025-09-22 02:28:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 459.781483ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [...................................X.XXXXXX..X..............] duration: 450.408985ms FAIL (2025-09-22 02:27:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== m30.954857157s) [seg_pos={0 0}, upload=496461]: Begin upload segment... Sep 22 02:15:55.602116061 (564h7m30.954901189s) [seg_pos={0 0}, upload=496461]: Done begin upload segment. Sep 22 02:15:55.602137471 (564h7m30.954922599s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602154613 (564h7m30.954939742s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602716609 (564h7m30.955501736s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602736295 (564h7m30.955521423s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603134873 (564h7m30.955920001s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603640462 (564h7m30.95642559s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604036745 (564h7m30.956821874s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604397323 (564h7m30.95718245s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604828131 (564h7m30.957613268s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605250674 (564h7m30.958035802s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605663548 (564h7m30.958448676s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606022221 (564h7m30.958807349s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606387297 (564h7m30.959172425s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606869060 (564h7m30.959654198s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607319626 (564h7m30.960104755s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607717662 (564h7m30.960502801s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608211139 (564h7m30.960996268s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608652857 (564h7m30.961437986s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.704831022 (564h7m31.05761613s) [upload=496461]: Next returned nil segment Sep 22 02:15:55.704846542 (564h7m31.05763165s) [upload=496461]: Waiting for error group managing segments... Sep 22 02:15:55.712709156 (564h7m31.065494285s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.713081926 (564h7m31.065867054s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.716351481 (564h7m31.06913661s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717069920 (564h7m31.069855048s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717303427 (564h7m31.070088556s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725190529 (564h7m31.077975657s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725273014 (564h7m31.078058142s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732220611 (564h7m31.085005749s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732337590 (564h7m31.085122719s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.734847580 (564h7m31.087632718s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.738226892 (564h7m31.09101203s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:16:55.564886089 (564h8m30.917671218s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1AowWwVvxzmnW5qwbqbMBVbvNoigffjAN51LQp8e6XA9CZkeXL, address:145.79.227.12:4790): protocol: rpc: dial tcp 145.79.227.12:4790: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564944549 (564h8m30.917729677s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564994583 (564h8m30.917779711s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565058042 (564h8m30.91784317s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1zywsXJnw5gfpUu7mC8p7UE7ev5HexPu5nSpUH1x2pzar99T97, address:145.79.227.8:4349): protocol: rpc: dial tcp 145.79.227.8:4349: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565089361 (564h8m30.917874489s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:145.79.227.14:4847): protocol: rpc: dial tcp 145.79.227.14:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565111382 (564h8m30.91789651s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12Pxos2xcjHwcYyQnXeRL3PL72G9pXaAnBs8XnkVFkxegDRPrqp, address:145.79.227.2:4127): protocol: rpc: dial tcp 145.79.227.2:4127: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565129266 (564h8m30.917914394s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12gs9LRE3oUp1jupCw5uDvRtkNcahbq2d4VTYQsdKr6atk6vGJx, address:145.79.227.19:4095): protocol: rpc: dial tcp 145.79.227.19:4095: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565148231 (564h8m30.917933359s) [seg_pos={0 0}, upload=496461]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:16:55.580701268 (564h8m30.933486366s) [upload=496461]: Done uploading segments... Sep 22 02:16:55.580714373 (564h8m30.93349947s) [upload=496461]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 450.437318ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [...............................XXX....XXX.XXXXX..X.XX.XX.XX.] duration: 428.448245ms FAIL (2025-09-22 02:27:33 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:26:47.079589262 (564h18m22.43237436s) [upload=496619]: Starting upload Sep 22 02:26:47.079635679 (564h18m22.432420787s) [upload=496619]: Done starting upload Sep 22 02:26:47.079683168 (564h18m22.432468266s) [upload=496619]: Uploading segments... Sep 22 02:26:47.079847085 (564h18m22.432632184s) [upload=496619]: Got next segment. Inline: false Sep 22 02:26:47.079880538 (564h18m22.432665637s) [upload=496619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:26:47.079884426 (564h18m22.432669524s) [upload=496619]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:26:47.098941027 (564h18m22.451726136s) [seg_pos={0 0}, upload=496619]: Begin upload segment... Sep 22 02:26:47.098967026 (564h18m22.451752124s) [seg_pos={0 0}, upload=496619]: Done begin upload segment. Sep 22 02:26:47.098988436 (564h18m22.451773544s) [noise=true , address=45.91.116.13:4153 , piece_id=KEVDRJJG3U53XZ42Y2N6JHAAWCI55B2LOTVXLTOLFRA7KLSIH6OA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099011058 (564h18m22.451796166s) [noise=true , address=145.79.227.2:4114 , piece_id=RIGKONJTORRHSKCZYTY5RHRKRWZG3ZPJLNP6MNFJENKINXUA3V7A, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099264234 (564h18m22.452049342s) [noise=true , address=45.91.116.9:4261 , piece_id=T36MVEHIYIU6GT23PS635SUF2K6OOU7JCSVKJ4TT6L43JEJV5CQA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099278360 (564h18m22.452063468s) [noise=true , address=145.79.227.19:4078 , piece_id=RDISZTZGXWT6C63QJ5A6ISV6YLTWMGFK43AZ37UN6TH7FB6IADUA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099503983 (564h18m22.452289081s) [noise=true , address=77.90.31.13:4626 , piece_id=NFHSBU7BRIFCYUOSOMPTJBUCJ5NBDHTRVJOET4W5IATUT6TMOYLA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099518230 (564h18m22.452303338s) [noise=true , address=145.79.227.6:4259 , piece_id=BBTEHIIGU2C7CTREQ7LF6GAIP66LMPQWS3CNHP5STLNOHSHRSALA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099716462 (564h18m22.452501561s) [noise=true , address=145.79.227.13:4804 , piece_id=TBHQSYEJUGV2XWHJIIPRWAYCLCMNKE7RKTPQY5PHDKNQJ5E5DA4A, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099730319 (564h18m22.452515417s) [noise=true , address=77.90.31.17:4771 , piece_id=MKFA2NQ6VFYBSYCYOQH7ILEZPYJ22CF3QGLQ5IMJYAPUXCAXQHUA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099964939 (564h18m22.452750047s) [noise=true , address=77.90.31.21:4878 , piece_id=64AOV3HKKUU6PCGD4SNDR3FVSGMTKKAMIV4JONEB5NIHQA2YJLIQ, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099968655 (564h18m22.452753754s) [noise=true , address=77.90.31.17:4770 , piece_id=RFZW4THLK3KTDNFJ25M3MXF2F3LD73APWI5EMZKBKIMQ72W6M65A, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100179582 (564h18m22.452964679s) [noise=true , address=45.91.116.8:4047 , piece_id=J4L2SJIV6VWR4O2VOH3PGNBPJIJG54QWRFKJAY72IWIL2UM3VBSA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100184851 (564h18m22.452969949s) [noise=true , address=77.90.31.14:4663 , piece_id=BHXXI4OQDYIR2E7PDGLEUYTLX5LRICO7THFGBS3BE4BCOWOX5NCA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100410064 (564h18m22.453195172s) [noise=true , address=77.90.31.15:4699 , piece_id=C25UYB5QNC7VOEDFN5TS26L424PTH7MQTESIQ4E24NLLHM35AABQ, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100477561 (564h18m22.453262669s) [noise=true , address=45.91.116.5:4297 , piece_id=YDYN4PVNPURJ6AHCQFVBTKVOLIZOIPKQYFJZ6GE2WR5TOEAW3IVQ, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100637391 (564h18m22.453422499s) [noise=true , address=45.91.116.4:4937 , piece_id=PL2JXVI6Q5A46PAIUGMJXRFQE4HYOPISL7KEE7CATBNLFABD75LQ, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100854929 (564h18m22.453640037s) [noise=true , address=45.91.116.6:4973 , piece_id=2KURM2DOGKAN3EDAZXIJDOPC6ACAE6GH7ORMQ45CV4TDN35DWE3Q, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.179812078 (564h18m22.532597186s) [upload=496619]: Next returned nil segment Sep 22 02:26:47.179824220 (564h18m22.532609328s) [upload=496619]: Waiting for error group managing segments... Sep 22 02:26:47.189480151 (564h18m22.542265249s) [noise=true , address=45.91.116.5:4297 , piece_id=YDYN4PVNPURJ6AHCQFVBTKVOLIZOIPKQYFJZ6GE2WR5TOEAW3IVQ, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.189604965 (564h18m22.542390064s) [noise=true , address=77.90.31.21:4878 , piece_id=64AOV3HKKUU6PCGD4SNDR3FVSGMTKKAMIV4JONEB5NIHQA2YJLIQ, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.189887455 (564h18m22.542672553s) [noise=true , address=45.91.116.8:4047 , piece_id=J4L2SJIV6VWR4O2VOH3PGNBPJIJG54QWRFKJAY72IWIL2UM3VBSA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.190226310 (564h18m22.543011409s) [noise=true , address=77.90.31.17:4770 , piece_id=RFZW4THLK3KTDNFJ25M3MXF2F3LD73APWI5EMZKBKIMQ72W6M65A, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.190460461 (564h18m22.543245559s) [noise=true , address=45.91.116.6:4973 , piece_id=2KURM2DOGKAN3EDAZXIJDOPC6ACAE6GH7ORMQ45CV4TDN35DWE3Q, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.191343909 (564h18m22.544129007s) [noise=true , address=45.91.116.9:4261 , piece_id=T36MVEHIYIU6GT23PS635SUF2K6OOU7JCSVKJ4TT6L43JEJV5CQA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.191584671 (564h18m22.544369779s) [noise=true , address=77.90.31.13:4626 , piece_id=NFHSBU7BRIFCYUOSOMPTJBUCJ5NBDHTRVJOET4W5IATUT6TMOYLA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.191710486 (564h18m22.544495584s) [noise=true , address=45.91.116.4:4937 , piece_id=PL2JXVI6Q5A46PAIUGMJXRFQE4HYOPISL7KEE7CATBNLFABD75LQ, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.192393257 (564h18m22.545178366s) [noise=true , address=77.90.31.17:4771 , piece_id=MKFA2NQ6VFYBSYCYOQH7ILEZPYJ22CF3QGLQ5IMJYAPUXCAXQHUA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.192411743 (564h18m22.54519684s) [noise=true , address=77.90.31.14:4663 , piece_id=BHXXI4OQDYIR2E7PDGLEUYTLX5LRICO7THFGBS3BE4BCOWOX5NCA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.192846608 (564h18m22.545631706s) [noise=true , address=45.91.116.13:4153 , piece_id=KEVDRJJG3U53XZ42Y2N6JHAAWCI55B2LOTVXLTOLFRA7KLSIH6OA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.194370378 (564h18m22.547155486s) [noise=true , address=77.90.31.15:4699 , piece_id=C25UYB5QNC7VOEDFN5TS26L424PTH7MQTESIQ4E24NLLHM35AABQ, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:27:17.067340274 (564h18m52.420125392s) [noise=true , address=145.79.227.2:4114 , piece_id=RIGKONJTORRHSKCZYTY5RHRKRWZG3ZPJLNP6MNFJENKINXUA3V7A, seg_pos={0 0}, upload=496619]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: rpc: dial tcp 145.79.227.2:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:27:17.067382292 (564h18m52.420167421s) [noise=true , address=145.79.227.19:4078 , piece_id=RDISZTZGXWT6C63QJ5A6ISV6YLTWMGFK43AZ37UN6TH7FB6IADUA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:27:17.067415285 (564h18m52.420200412s) [noise=true , address=145.79.227.13:4804 , piece_id=TBHQSYEJUGV2XWHJIIPRWAYCLCMNKE7RKTPQY5PHDKNQJ5E5DA4A, seg_pos={0 0}, upload=496619]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:27:17.067426626 (564h18m52.420211754s) [noise=true , address=145.79.227.6:4259 , piece_id=BBTEHIIGU2C7CTREQ7LF6GAIP66LMPQWS3CNHP5STLNOHSHRSALA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: rpc: dial tcp 145.79.227.6:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:27:17.067470808 (564h18m52.420255937s) [seg_pos={0 0}, upload=496619]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:27:17.079419761 (564h18m52.43220488s) [upload=496619]: Done uploading segments... Sep 22 02:27:17.079433988 (564h18m52.432219117s) [upload=496619]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 428.466899ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [................................XXX....XXX.XXXXX..X.XX.XX.XX] duration: 30.013534026s FAIL (2025-09-22 02:27:17 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:26:47.079589262 (564h18m22.43237436s) [upload=496619]: Starting upload Sep 22 02:26:47.079635679 (564h18m22.432420787s) [upload=496619]: Done starting upload Sep 22 02:26:47.079683168 (564h18m22.432468266s) [upload=496619]: Uploading segments... Sep 22 02:26:47.079847085 (564h18m22.432632184s) [upload=496619]: Got next segment. Inline: false Sep 22 02:26:47.079880538 (564h18m22.432665637s) [upload=496619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:26:47.079884426 (564h18m22.432669524s) [upload=496619]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:26:47.098941027 (564h18m22.451726136s) [seg_pos={0 0}, upload=496619]: Begin upload segment... Sep 22 02:26:47.098967026 (564h18m22.451752124s) [seg_pos={0 0}, upload=496619]: Done begin upload segment. Sep 22 02:26:47.098988436 (564h18m22.451773544s) [noise=true , address=45.91.116.13:4153 , piece_id=KEVDRJJG3U53XZ42Y2N6JHAAWCI55B2LOTVXLTOLFRA7KLSIH6OA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099011058 (564h18m22.451796166s) [noise=true , address=145.79.227.2:4114 , piece_id=RIGKONJTORRHSKCZYTY5RHRKRWZG3ZPJLNP6MNFJENKINXUA3V7A, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099264234 (564h18m22.452049342s) [noise=true , address=45.91.116.9:4261 , piece_id=T36MVEHIYIU6GT23PS635SUF2K6OOU7JCSVKJ4TT6L43JEJV5CQA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099278360 (564h18m22.452063468s) [noise=true , address=145.79.227.19:4078 , piece_id=RDISZTZGXWT6C63QJ5A6ISV6YLTWMGFK43AZ37UN6TH7FB6IADUA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099503983 (564h18m22.452289081s) [noise=true , address=77.90.31.13:4626 , piece_id=NFHSBU7BRIFCYUOSOMPTJBUCJ5NBDHTRVJOET4W5IATUT6TMOYLA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099518230 (564h18m22.452303338s) [noise=true , address=145.79.227.6:4259 , piece_id=BBTEHIIGU2C7CTREQ7LF6GAIP66LMPQWS3CNHP5STLNOHSHRSALA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099716462 (564h18m22.452501561s) [noise=true , address=145.79.227.13:4804 , piece_id=TBHQSYEJUGV2XWHJIIPRWAYCLCMNKE7RKTPQY5PHDKNQJ5E5DA4A, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099730319 (564h18m22.452515417s) [noise=true , address=77.90.31.17:4771 , piece_id=MKFA2NQ6VFYBSYCYOQH7ILEZPYJ22CF3QGLQ5IMJYAPUXCAXQHUA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099964939 (564h18m22.452750047s) [noise=true , address=77.90.31.21:4878 , piece_id=64AOV3HKKUU6PCGD4SNDR3FVSGMTKKAMIV4JONEB5NIHQA2YJLIQ, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.099968655 (564h18m22.452753754s) [noise=true , address=77.90.31.17:4770 , piece_id=RFZW4THLK3KTDNFJ25M3MXF2F3LD73APWI5EMZKBKIMQ72W6M65A, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100179582 (564h18m22.452964679s) [noise=true , address=45.91.116.8:4047 , piece_id=J4L2SJIV6VWR4O2VOH3PGNBPJIJG54QWRFKJAY72IWIL2UM3VBSA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100184851 (564h18m22.452969949s) [noise=true , address=77.90.31.14:4663 , piece_id=BHXXI4OQDYIR2E7PDGLEUYTLX5LRICO7THFGBS3BE4BCOWOX5NCA, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100410064 (564h18m22.453195172s) [noise=true , address=77.90.31.15:4699 , piece_id=C25UYB5QNC7VOEDFN5TS26L424PTH7MQTESIQ4E24NLLHM35AABQ, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100477561 (564h18m22.453262669s) [noise=true , address=45.91.116.5:4297 , piece_id=YDYN4PVNPURJ6AHCQFVBTKVOLIZOIPKQYFJZ6GE2WR5TOEAW3IVQ, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100637391 (564h18m22.453422499s) [noise=true , address=45.91.116.4:4937 , piece_id=PL2JXVI6Q5A46PAIUGMJXRFQE4HYOPISL7KEE7CATBNLFABD75LQ, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.100854929 (564h18m22.453640037s) [noise=true , address=45.91.116.6:4973 , piece_id=2KURM2DOGKAN3EDAZXIJDOPC6ACAE6GH7ORMQ45CV4TDN35DWE3Q, seg_pos={0 0}, upload=496619]: Uploading piece... Sep 22 02:26:47.179812078 (564h18m22.532597186s) [upload=496619]: Next returned nil segment Sep 22 02:26:47.179824220 (564h18m22.532609328s) [upload=496619]: Waiting for error group managing segments... Sep 22 02:26:47.189480151 (564h18m22.542265249s) [noise=true , address=45.91.116.5:4297 , piece_id=YDYN4PVNPURJ6AHCQFVBTKVOLIZOIPKQYFJZ6GE2WR5TOEAW3IVQ, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.189604965 (564h18m22.542390064s) [noise=true , address=77.90.31.21:4878 , piece_id=64AOV3HKKUU6PCGD4SNDR3FVSGMTKKAMIV4JONEB5NIHQA2YJLIQ, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.189887455 (564h18m22.542672553s) [noise=true , address=45.91.116.8:4047 , piece_id=J4L2SJIV6VWR4O2VOH3PGNBPJIJG54QWRFKJAY72IWIL2UM3VBSA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.190226310 (564h18m22.543011409s) [noise=true , address=77.90.31.17:4770 , piece_id=RFZW4THLK3KTDNFJ25M3MXF2F3LD73APWI5EMZKBKIMQ72W6M65A, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.190460461 (564h18m22.543245559s) [noise=true , address=45.91.116.6:4973 , piece_id=2KURM2DOGKAN3EDAZXIJDOPC6ACAE6GH7ORMQ45CV4TDN35DWE3Q, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.191343909 (564h18m22.544129007s) [noise=true , address=45.91.116.9:4261 , piece_id=T36MVEHIYIU6GT23PS635SUF2K6OOU7JCSVKJ4TT6L43JEJV5CQA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.191584671 (564h18m22.544369779s) [noise=true , address=77.90.31.13:4626 , piece_id=NFHSBU7BRIFCYUOSOMPTJBUCJ5NBDHTRVJOET4W5IATUT6TMOYLA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.191710486 (564h18m22.544495584s) [noise=true , address=45.91.116.4:4937 , piece_id=PL2JXVI6Q5A46PAIUGMJXRFQE4HYOPISL7KEE7CATBNLFABD75LQ, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.192393257 (564h18m22.545178366s) [noise=true , address=77.90.31.17:4771 , piece_id=MKFA2NQ6VFYBSYCYOQH7ILEZPYJ22CF3QGLQ5IMJYAPUXCAXQHUA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.192411743 (564h18m22.54519684s) [noise=true , address=77.90.31.14:4663 , piece_id=BHXXI4OQDYIR2E7PDGLEUYTLX5LRICO7THFGBS3BE4BCOWOX5NCA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.192846608 (564h18m22.545631706s) [noise=true , address=45.91.116.13:4153 , piece_id=KEVDRJJG3U53XZ42Y2N6JHAAWCI55B2LOTVXLTOLFRA7KLSIH6OA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:26:47.194370378 (564h18m22.547155486s) [noise=true , address=77.90.31.15:4699 , piece_id=C25UYB5QNC7VOEDFN5TS26L424PTH7MQTESIQ4E24NLLHM35AABQ, seg_pos={0 0}, upload=496619]: Done uploading piece. err: <nil> Sep 22 02:27:17.067340274 (564h18m52.420125392s) [noise=true , address=145.79.227.2:4114 , piece_id=RIGKONJTORRHSKCZYTY5RHRKRWZG3ZPJLNP6MNFJENKINXUA3V7A, seg_pos={0 0}, upload=496619]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: rpc: dial tcp 145.79.227.2:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:27:17.067382292 (564h18m52.420167421s) [noise=true , address=145.79.227.19:4078 , piece_id=RDISZTZGXWT6C63QJ5A6ISV6YLTWMGFK43AZ37UN6TH7FB6IADUA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:27:17.067415285 (564h18m52.420200412s) [noise=true , address=145.79.227.13:4804 , piece_id=TBHQSYEJUGV2XWHJIIPRWAYCLCMNKE7RKTPQY5PHDKNQJ5E5DA4A, seg_pos={0 0}, upload=496619]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:27:17.067426626 (564h18m52.420211754s) [noise=true , address=145.79.227.6:4259 , piece_id=BBTEHIIGU2C7CTREQ7LF6GAIP66LMPQWS3CNHP5STLNOHSHRSALA, seg_pos={0 0}, upload=496619]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: rpc: dial tcp 145.79.227.6:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:27:17.067470808 (564h18m52.420255937s) [seg_pos={0 0}, upload=496619]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:27:17.079419761 (564h18m52.43220488s) [upload=496619]: Done uploading segments... Sep 22 02:27:17.079433988 (564h18m52.432219117s) [upload=496619]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.013554795s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [....................................X.XXXXXX..X.............] duration: 679.958389ms FAIL (2025-09-22 02:26:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== m30.954857157s) [seg_pos={0 0}, upload=496461]: Begin upload segment... Sep 22 02:15:55.602116061 (564h7m30.954901189s) [seg_pos={0 0}, upload=496461]: Done begin upload segment. Sep 22 02:15:55.602137471 (564h7m30.954922599s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602154613 (564h7m30.954939742s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602716609 (564h7m30.955501736s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602736295 (564h7m30.955521423s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603134873 (564h7m30.955920001s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603640462 (564h7m30.95642559s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604036745 (564h7m30.956821874s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604397323 (564h7m30.95718245s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604828131 (564h7m30.957613268s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605250674 (564h7m30.958035802s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605663548 (564h7m30.958448676s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606022221 (564h7m30.958807349s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606387297 (564h7m30.959172425s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606869060 (564h7m30.959654198s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607319626 (564h7m30.960104755s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607717662 (564h7m30.960502801s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608211139 (564h7m30.960996268s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608652857 (564h7m30.961437986s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.704831022 (564h7m31.05761613s) [upload=496461]: Next returned nil segment Sep 22 02:15:55.704846542 (564h7m31.05763165s) [upload=496461]: Waiting for error group managing segments... Sep 22 02:15:55.712709156 (564h7m31.065494285s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.713081926 (564h7m31.065867054s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.716351481 (564h7m31.06913661s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717069920 (564h7m31.069855048s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717303427 (564h7m31.070088556s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725190529 (564h7m31.077975657s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725273014 (564h7m31.078058142s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732220611 (564h7m31.085005749s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732337590 (564h7m31.085122719s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.734847580 (564h7m31.087632718s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.738226892 (564h7m31.09101203s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:16:55.564886089 (564h8m30.917671218s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1AowWwVvxzmnW5qwbqbMBVbvNoigffjAN51LQp8e6XA9CZkeXL, address:145.79.227.12:4790): protocol: rpc: dial tcp 145.79.227.12:4790: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564944549 (564h8m30.917729677s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564994583 (564h8m30.917779711s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565058042 (564h8m30.91784317s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1zywsXJnw5gfpUu7mC8p7UE7ev5HexPu5nSpUH1x2pzar99T97, address:145.79.227.8:4349): protocol: rpc: dial tcp 145.79.227.8:4349: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565089361 (564h8m30.917874489s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:145.79.227.14:4847): protocol: rpc: dial tcp 145.79.227.14:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565111382 (564h8m30.91789651s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12Pxos2xcjHwcYyQnXeRL3PL72G9pXaAnBs8XnkVFkxegDRPrqp, address:145.79.227.2:4127): protocol: rpc: dial tcp 145.79.227.2:4127: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565129266 (564h8m30.917914394s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12gs9LRE3oUp1jupCw5uDvRtkNcahbq2d4VTYQsdKr6atk6vGJx, address:145.79.227.19:4095): protocol: rpc: dial tcp 145.79.227.19:4095: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565148231 (564h8m30.917933359s) [seg_pos={0 0}, upload=496461]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:16:55.580701268 (564h8m30.933486366s) [upload=496461]: Done uploading segments... Sep 22 02:16:55.580714373 (564h8m30.93349947s) [upload=496461]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 679.988595ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [.................................XXX....XXX.XXXXX..X.XX.XX.X] duration: 30.019080807s FAIL (2025-09-22 02:26:32 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Sep 22 02:26:02.232056841 (564h17m37.584841949s) [upload=496607]: Starting upload Sep 22 02:26:02.232105683 (564h17m37.584890791s) [upload=496607]: Done starting upload Sep 22 02:26:02.232200871 (564h17m37.584985979s) [upload=496607]: Uploading segments... Sep 22 02:26:02.232407819 (564h17m37.585192918s) [upload=496607]: Got next segment. Inline: false Sep 22 02:26:02.232438767 (564h17m37.585223866s) [upload=496607]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:26:02.232480516 (564h17m37.585265624s) [upload=496607]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:26:02.257104705 (564h17m37.609889804s) [seg_pos={0 0}, upload=496607]: Begin upload segment... Sep 22 02:26:02.257143629 (564h17m37.609928737s) [seg_pos={0 0}, upload=496607]: Done begin upload segment. Sep 22 02:26:02.257162584 (564h17m37.609947693s) [noise=true , address=45.91.116.13:4153 , piece_id=73XDQIP6Z3VNVZY5GS4XOOVVPPACPZVP5R4WQ66CF3GCMQG7VI5A, seg_pos={0 0}, upload=496607]: Uploading piece... Sep 22 02:26:02.257180487 (564h17m37.609965606s) [noise=true , address=145.79.227.7:4294 , piece_id=NKQX33RLMLRX4TNVLTQ7NICQZWUBBE6FDDYTTLCO72T5NZQOGYGQ, seg_pos={0 0}, upload=496607]: Uploading piece... Sep 22 02:26:02.257500999 (564h17m37.610286107s) [noise=true , address=45.91.116.4:4937 , piece_id=Y67SDYW3OS2Z6TBUEBYBDZRUR52FLEHV4CEHFLPNNNDE2MRUX4VQ, seg_pos={0 0}, upload=496607]: Uploading piece... Sep 22 02:26:02.257598551 (564h17m37.61038366s) [noise=true , address=77.90.31.18:4806 , piece_id=ZTMWWSSESXYIATNKAKKUHF36RK5O2NGEJJEXSHPTEVT5VU2SXQTA, seg_pos={0 0}, upload=496607]: Uploading piece... Sep 22 02:26:02.257752972 (564h17m37.610538069s) [noise=true , address=145.79.227.2:4113 , piece_id=EMRMIX4ELLD774DXNHYBOPGT25ACMYW2KHCCW4N72K2TKIAULW3Q, seg_pos={0 0}, upload=496607]: Uploading piece... Sep 22 02:26:02.257952307 (564h17m37.610737415s) [noise=true , address=45.91.116.5:4299 , piece_id=MSNFVE3UY3LJFQJK6IFLE7UG72NL3XOFOSKF5SC4OPVSR2TJXGFQ, seg_pos={0 0}, upload=496607]: Uploading piece... Sep 22 02:26:02.257984857 (564h17m37.610769965s) [noise=true , address=77.90.31.17:4772 , piece_id=MXPLO3XHFW6YS652U2QRBCLXQRU7FG26P6GBXPLSRSXXZOSA2OYQ, seg_pos={0 0}, upload=496607]: Uploading piece... Sep 22 02:26:02.258217003 (564h17m37.6110021s) [noise=true , address=77.90.31.19:4841 , piece_id=FDITHAF4LCFY6LPMIR4PY5UF3NNBF6ELR32KUWHSIVEKAKNEBZHQ, seg_pos={0 0}, upload=496607]: Uploading piece... Sep 22 02:26:02.258327390 (564h17m37.611112498s) [noise=true , address=145.79.227.13:4801 , piece_id=MRFGHSAH3NPY4O2KXWMKMZMIETYQDHOK6E4MRSI7GJ3UU66YYS5Q, seg_pos={0 0}, upload=496607]: Uploading piece... Sep 22 02:26:02.258462523 (564h17m37.611247621s) [noise=true , address=145.79.227.17:4969 , piece_id=2BMCLOA3IMGNB6ADJX53QTBWR2MA6QS3DPGQZOEODWDC52UTLIMQ, seg_pos={0 0}, upload=496607]: Uploading piece... Sep 22 02:26:02.258712482 (564h17m37.61149759s) [noise=true , address=45.91.116.11:4118 , piece_id=IGJE2JQ4MRVQLML6QFVSCJ6ACVPUBMCBUWRRKRQG33S7GC7G3VMA, seg_pos={0 0}, upload=496607]: Uploading piece... Sep 22 02:26:02.258764470 (564h17m37.611549578s) [noise=true , address=45.91.116.12:4226 , piece_id=YXUG4SL5737PHQR5OEN27VLAL4WIA5LEZARM57FG2JPHXCSNVURQ, seg_pos={0 0}, upload=496607]: Uploading piece... Sep 22 02:26:02.258948015 (564h17m37.611733123s) [noise=true , address=77.90.31.15:4698 , piece_id=I4AOYFUBY4GGEBKHCFRCCJ2SNURJRGJZQR5IJYSO66FBMMQHOYWQ, seg_pos={0 0}, upload=496607]: Uploading piece... Sep 22 02:26:02.259051810 (564h17m37.611836907s) [noise=true , address=145.79.227.17:4972 , piece_id=6UVQURS5N3DH7ACVTWMLOTYXDGVAIEDEI5TBS72WZGVYVOJEWITA, seg_pos={0 0}, upload=496607]: Uploading piece... Sep 22 02:26:02.259299173 (564h17m37.612084271s) [noise=true , address=145.79.227.14:4839 , piece_id=K3CV6ECUEKUVOBCK4F23M6YZHXQMUGFRN3ITBNYGFJRW7GNIYZTQ, seg_pos={0 0}, upload=496607]: Uploading piece... Sep 22 02:26:02.259554803 (564h17m37.612339901s) [noise=true , address=145.79.227.19:4077 , piece_id=MLQAOFHH44SD4WMLAA2T2PFSQMKJTRIXNZN5JZHKBTQKU6BOW5OQ, seg_pos={0 0}, upload=496607]: Uploading piece... Sep 22 02:26:02.332878651 (564h17m37.685663749s) [upload=496607]: Next returned nil segment Sep 22 02:26:02.332898629 (564h17m37.685683737s) [upload=496607]: Waiting for error group managing segments... Sep 22 02:26:02.342060822 (564h17m37.69484593s) [noise=true , address=77.90.31.18:4806 , piece_id=ZTMWWSSESXYIATNKAKKUHF36RK5O2NGEJJEXSHPTEVT5VU2SXQTA, seg_pos={0 0}, upload=496607]: Done uploading piece. err: <nil> Sep 22 02:26:02.343082860 (564h17m37.695867957s) [noise=true , address=45.91.116.11:4118 , piece_id=IGJE2JQ4MRVQLML6QFVSCJ6ACVPUBMCBUWRRKRQG33S7GC7G3VMA, seg_pos={0 0}, upload=496607]: Done uploading piece. err: <nil> Sep 22 02:26:02.343822237 (564h17m37.696607335s) [noise=true , address=45.91.116.13:4153 , piece_id=73XDQIP6Z3VNVZY5GS4XOOVVPPACPZVP5R4WQ66CF3GCMQG7VI5A, seg_pos={0 0}, upload=496607]: Done uploading piece. err: <nil> Sep 22 02:26:02.344167615 (564h17m37.696952723s) [noise=true , address=45.91.116.4:4937 , piece_id=Y67SDYW3OS2Z6TBUEBYBDZRUR52FLEHV4CEHFLPNNNDE2MRUX4VQ, seg_pos={0 0}, upload=496607]: Done uploading piece. err: <nil> Sep 22 02:26:02.344177924 (564h17m37.696963032s) [noise=true , address=45.91.116.5:4299 , piece_id=MSNFVE3UY3LJFQJK6IFLE7UG72NL3XOFOSKF5SC4OPVSR2TJXGFQ, seg_pos={0 0}, upload=496607]: Done uploading piece. err: <nil> Sep 22 02:26:02.344665530 (564h17m37.697450638s) [noise=true , address=77.90.31.19:4841 , piece_id=FDITHAF4LCFY6LPMIR4PY5UF3NNBF6ELR32KUWHSIVEKAKNEBZHQ, seg_pos={0 0}, upload=496607]: Done uploading piece. err: <nil> Sep 22 02:26:02.344687952 (564h17m37.69747306s) [noise=true , address=45.91.116.12:4226 , piece_id=YXUG4SL5737PHQR5OEN27VLAL4WIA5LEZARM57FG2JPHXCSNVURQ, seg_pos={0 0}, upload=496607]: Done uploading piece. err: <nil> Sep 22 02:26:02.346999719 (564h17m37.699784827s) [noise=true , address=77.90.31.15:4698 , piece_id=I4AOYFUBY4GGEBKHCFRCCJ2SNURJRGJZQR5IJYSO66FBMMQHOYWQ, seg_pos={0 0}, upload=496607]: Done uploading piece. err: <nil> Sep 22 02:26:02.347402696 (564h17m37.700187804s) [noise=true , address=77.90.31.17:4772 , piece_id=MXPLO3XHFW6YS652U2QRBCLXQRU7FG26P6GBXPLSRSXXZOSA2OYQ, seg_pos={0 0}, upload=496607]: Done uploading piece. err: <nil> Sep 22 02:26:32.220713405 (564h18m7.573498543s) [noise=true , address=145.79.227.19:4077 , piece_id=MLQAOFHH44SD4WMLAA2T2PFSQMKJTRIXNZN5JZHKBTQKU6BOW5OQ, seg_pos={0 0}, upload=496607]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:145.79.227.19:4077): protocol: rpc: dial tcp 145.79.227.19:4077: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:26:32.220774309 (564h18m7.573559438s) [noise=true , address=145.79.227.7:4294 , piece_id=NKQX33RLMLRX4TNVLTQ7NICQZWUBBE6FDDYTTLCO72T5NZQOGYGQ, seg_pos={0 0}, upload=496607]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: rpc: dial tcp 145.79.227.7:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:26:32.220815477 (564h18m7.573600606s) [noise=true , address=145.79.227.13:4801 , piece_id=MRFGHSAH3NPY4O2KXWMKMZMIETYQDHOK6E4MRSI7GJ3UU66YYS5Q, seg_pos={0 0}, upload=496607]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: rpc: dial tcp 145.79.227.13:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:26:32.220951693 (564h18m7.573736831s) [noise=true , address=145.79.227.17:4969 , piece_id=2BMCLOA3IMGNB6ADJX53QTBWR2MA6QS3DPGQZOEODWDC52UTLIMQ, seg_pos={0 0}, upload=496607]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:145.79.227.17:4969): protocol: rpc: dial tcp 145.79.227.17:4969: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:26:32.221015784 (564h18m7.573800911s) [noise=true , address=145.79.227.14:4839 , piece_id=K3CV6ECUEKUVOBCK4F23M6YZHXQMUGFRN3ITBNYGFJRW7GNIYZTQ, seg_pos={0 0}, upload=496607]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: rpc: dial tcp 145.79.227.14:4839: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:26:32.221058082 (564h18m7.573843211s) [noise=true , address=145.79.227.17:4972 , piece_id=6UVQURS5N3DH7ACVTWMLOTYXDGVAIEDEI5TBS72WZGVYVOJEWITA, seg_pos={0 0}, upload=496607]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: rpc: dial tcp 145.79.227.17:4972: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:26:32.221075215 (564h18m7.573860352s) [noise=true , address=145.79.227.2:4113 , piece_id=EMRMIX4ELLD774DXNHYBOPGT25ACMYW2KHCCW4N72K2TKIAULW3Q, seg_pos={0 0}, upload=496607]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: rpc: dial tcp 145.79.227.2:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:26:32.221114088 (564h18m7.573899225s) [seg_pos={0 0}, upload=496607]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 22 02:26:32.236294484 (564h18m7.589079613s) [upload=496607]: Done uploading segments... Sep 22 02:26:32.236310114 (564h18m7.589095242s) [upload=496607]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.019105524s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................................X.X.........X.......X...] duration: 687.078371ms FAIL (2025-09-22 02:26:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 687.09916ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [.....................................X.XXXXXX..X............] duration: 498.404981ms FAIL (2025-09-22 02:26:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== m30.954857157s) [seg_pos={0 0}, upload=496461]: Begin upload segment... Sep 22 02:15:55.602116061 (564h7m30.954901189s) [seg_pos={0 0}, upload=496461]: Done begin upload segment. Sep 22 02:15:55.602137471 (564h7m30.954922599s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602154613 (564h7m30.954939742s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602716609 (564h7m30.955501736s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602736295 (564h7m30.955521423s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603134873 (564h7m30.955920001s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603640462 (564h7m30.95642559s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604036745 (564h7m30.956821874s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604397323 (564h7m30.95718245s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604828131 (564h7m30.957613268s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605250674 (564h7m30.958035802s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605663548 (564h7m30.958448676s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606022221 (564h7m30.958807349s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606387297 (564h7m30.959172425s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606869060 (564h7m30.959654198s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607319626 (564h7m30.960104755s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607717662 (564h7m30.960502801s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608211139 (564h7m30.960996268s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608652857 (564h7m30.961437986s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.704831022 (564h7m31.05761613s) [upload=496461]: Next returned nil segment Sep 22 02:15:55.704846542 (564h7m31.05763165s) [upload=496461]: Waiting for error group managing segments... Sep 22 02:15:55.712709156 (564h7m31.065494285s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.713081926 (564h7m31.065867054s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.716351481 (564h7m31.06913661s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717069920 (564h7m31.069855048s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717303427 (564h7m31.070088556s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725190529 (564h7m31.077975657s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725273014 (564h7m31.078058142s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732220611 (564h7m31.085005749s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732337590 (564h7m31.085122719s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.734847580 (564h7m31.087632718s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.738226892 (564h7m31.09101203s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:16:55.564886089 (564h8m30.917671218s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1AowWwVvxzmnW5qwbqbMBVbvNoigffjAN51LQp8e6XA9CZkeXL, address:145.79.227.12:4790): protocol: rpc: dial tcp 145.79.227.12:4790: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564944549 (564h8m30.917729677s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564994583 (564h8m30.917779711s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565058042 (564h8m30.91784317s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1zywsXJnw5gfpUu7mC8p7UE7ev5HexPu5nSpUH1x2pzar99T97, address:145.79.227.8:4349): protocol: rpc: dial tcp 145.79.227.8:4349: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565089361 (564h8m30.917874489s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:145.79.227.14:4847): protocol: rpc: dial tcp 145.79.227.14:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565111382 (564h8m30.91789651s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12Pxos2xcjHwcYyQnXeRL3PL72G9pXaAnBs8XnkVFkxegDRPrqp, address:145.79.227.2:4127): protocol: rpc: dial tcp 145.79.227.2:4127: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565129266 (564h8m30.917914394s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12gs9LRE3oUp1jupCw5uDvRtkNcahbq2d4VTYQsdKr6atk6vGJx, address:145.79.227.19:4095): protocol: rpc: dial tcp 145.79.227.19:4095: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565148231 (564h8m30.917933359s) [seg_pos={0 0}, upload=496461]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:16:55.580701268 (564h8m30.933486366s) [upload=496461]: Done uploading segments... Sep 22 02:16:55.580714373 (564h8m30.93349947s) [upload=496461]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 498.434195ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................................X.X.........X.......X..] duration: 405.532779ms FAIL (2025-09-22 02:26:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 405.552025ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................................X.X.........X.......X.] duration: 437.027313ms FAIL (2025-09-22 02:25:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 437.059323ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [..................................XXX....XXX.XXXXX..X.XX.XX.] duration: 551.816422ms FAIL (2025-09-22 02:25:05 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:24:29.913646711 (564h16m5.266431849s) [upload=496585]: Starting upload Sep 22 02:24:29.913754343 (564h16m5.266539481s) [upload=496585]: Done starting upload Sep 22 02:24:29.913826157 (564h16m5.266611296s) [upload=496585]: Uploading segments... Sep 22 02:24:29.914160415 (564h16m5.266945553s) [upload=496585]: Got next segment. Inline: false Sep 22 02:24:29.914231738 (564h16m5.267016866s) [upload=496585]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:24:29.914241637 (564h16m5.267026775s) [upload=496585]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:24:29.943512376 (564h16m5.296297504s) [seg_pos={0 0}, upload=496585]: Begin upload segment... Sep 22 02:24:29.943553513 (564h16m5.296338642s) [seg_pos={0 0}, upload=496585]: Done begin upload segment. Sep 22 02:24:29.943582378 (564h16m5.296367515s) [noise=true , address=45.91.116.9:4263 , piece_id=LPPLXP5T6REENEO7W2HT7ZSUI3ZHYEIPG37OHXORDF5B3Y6MGWUA, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.943612053 (564h16m5.296397182s) [noise=true , address=145.79.227.19:4078 , piece_id=O3SCR4MJDILV6JSGNN7CK4TIMQED4HEHFZMV5FFIWL7E5HD35R5A, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.944105910 (564h16m5.296891048s) [noise=true , address=77.90.31.21:4879 , piece_id=SOOFFNV3GFMFI7VXR5OJEHCGUHOQEP2XGS2SSSRTWDJDQV2QGFEA, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.944149352 (564h16m5.29693448s) [noise=true , address=145.79.227.11:4732 , piece_id=4BG4QVSKDMAU5WTQMHVQ3GMPFSHVW2CEIHHPV6R3BLQZA26KZLJA, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.944537811 (564h16m5.297322938s) [noise=true , address=77.90.31.11:4554 , piece_id=WAICDOYFSQTF2SYFYTZ2KPCISRSINU4HVJ24SVR5Q4QKOCY4QD2A, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.944550444 (564h16m5.297335583s) [noise=true , address=45.91.116.14:4190 , piece_id=3PZ6EV4HUPZGBMYI7M3X5C27KV4T6UNH62J2365D5RHPRQUUMZHA, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.944923614 (564h16m5.297708782s) [noise=true , address=45.91.116.4:4937 , piece_id=TWAKWSJ6THZF2P6E4EHSWOCNZCJGFJMYLCAWJ2UD6WTT63KBST5A, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.944966284 (564h16m5.297751422s) [noise=true , address=77.90.31.21:4877 , piece_id=BD6HJWSXDSDFBNRCQFYGSUYSFLDQDKAOT6AIFZBNM554YXIHUDAA, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.945367027 (564h16m5.298152165s) [noise=true , address=145.79.227.17:4971 , piece_id=KJA4H3RCIJFCLDYPF4VCSKBDGPC3RLDBVP37KRDEY44WA4B2YRNA, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.945403825 (564h16m5.298188954s) [noise=true , address=77.90.31.13:4627 , piece_id=3PDJWOWQ5776HPH2XHYWDBNSFX3CENHSBKXF4MTQYVAHI4MG2OMQ, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.945971571 (564h16m5.2987567s) [noise=true , address=145.79.227.5:4223 , piece_id=G537GPGFD4JY7HEVFTQS64JZ5HFGBNSYIMNSN2ZVL6J2QQDE7XIA, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.946404894 (564h16m5.299190023s) [noise=true , address=77.90.31.12:4589 , piece_id=DXJ7BQZETSXWHRX7K5D366ADUYJL5Q7EK3C65MCCN2KDD6XTNDMA, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.946622683 (564h16m5.299407811s) [noise=true , address=45.91.116.7:4010 , piece_id=BKVKLHYYXAPXUM2NDIAFUF7THYMNNFY4WSXMDTDN3NNXJ7XBQR4A, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.946886869 (564h16m5.299672048s) [noise=true , address=45.91.116.12:4226 , piece_id=MOD26FVJAHEO6CXLKDFCQH4FQWJXUGQGLU44WYIPHFJDYJPPFLGQ, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.947149081 (564h16m5.299934209s) [noise=true , address=77.90.31.18:4807 , piece_id=NT4UEGNET7FPI2HMLPDAFUIENECL6JG7LFA3ADPXQTP3KLGPMAQA, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.947312267 (564h16m5.300097395s) [noise=true , address=45.91.116.8:4047 , piece_id=WB7UZ72ATOKXWK3DGQ2J2PE3SE5Z3W77QR77GMPN6YMXK5KUDLBQ, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:30.082454071 (564h16m5.435239189s) [upload=496585]: Next returned nil segment Sep 22 02:24:30.082468158 (564h16m5.435253276s) [upload=496585]: Waiting for error group managing segments... Sep 22 02:24:30.091223908 (564h16m5.444009016s) [noise=true , address=45.91.116.12:4226 , piece_id=MOD26FVJAHEO6CXLKDFCQH4FQWJXUGQGLU44WYIPHFJDYJPPFLGQ, seg_pos={0 0}, upload=496585]: Done uploading piece. err: <nil> Sep 22 02:24:30.091566551 (564h16m5.444351679s) [noise=true , address=45.91.116.7:4010 , piece_id=BKVKLHYYXAPXUM2NDIAFUF7THYMNNFY4WSXMDTDN3NNXJ7XBQR4A, seg_pos={0 0}, upload=496585]: Done uploading piece. err: <nil> Sep 22 02:24:30.091665817 (564h16m5.444450925s) [noise=true , address=77.90.31.21:4879 , piece_id=SOOFFNV3GFMFI7VXR5OJEHCGUHOQEP2XGS2SSSRTWDJDQV2QGFEA, seg_pos={0 0}, upload=496585]: Done uploading piece. err: <nil> Sep 22 02:24:30.092516043 (564h16m5.445301171s) [noise=true , address=45.91.116.9:4263 , piece_id=LPPLXP5T6REENEO7W2HT7ZSUI3ZHYEIPG37OHXORDF5B3Y6MGWUA, seg_pos={0 0}, upload=496585]: Done uploading piece. err: <nil> Sep 22 02:24:30.093580169 (564h16m5.446365288s) [noise=true , address=77.90.31.12:4589 , piece_id=DXJ7BQZETSXWHRX7K5D366ADUYJL5Q7EK3C65MCCN2KDD6XTNDMA, seg_pos={0 0}, upload=496585]: Done uploading piece. err: <nil> Sep 22 02:24:30.094634559 (564h16m5.447419677s) [noise=true , address=77.90.31.18:4807 , piece_id=NT4UEGNET7FPI2HMLPDAFUIENECL6JG7LFA3ADPXQTP3KLGPMAQA, seg_pos={0 0}, upload=496585]: Done uploading piece. err: <nil> Sep 22 02:24:30.095551219 (564h16m5.448336348s) [noise=true , address=77.90.31.11:4554 , piece_id=WAICDOYFSQTF2SYFYTZ2KPCISRSINU4HVJ24SVR5Q4QKOCY4QD2A, seg_pos={0 0}, upload=496585]: Done uploading piece. err: <nil> Sep 22 02:24:30.096503045 (564h16m5.449288174s) [noise=true , address=77.90.31.21:4877 , piece_id=BD6HJWSXDSDFBNRCQFYGSUYSFLDQDKAOT6AIFZBNM554YXIHUDAA, seg_pos={0 0}, upload=496585]: Done uploading piece. err: <nil> Sep 22 02:24:30.097369932 (564h16m5.450155059s) [noise=true , address=45.91.116.8:4047 , piece_id=WB7UZ72ATOKXWK3DGQ2J2PE3SE5Z3W77QR77GMPN6YMXK5KUDLBQ, seg_pos={0 0}, upload=496585]: Done uploading piece. err: <nil> Sep 22 02:24:30.097909815 (564h16m5.450694943s) [noise=true , address=45.91.116.14:4190 , piece_id=3PZ6EV4HUPZGBMYI7M3X5C27KV4T6UNH62J2365D5RHPRQUUMZHA, seg_pos={0 0}, upload=496585]: Done uploading piece. err: <nil> Sep 22 02:24:30.099080552 (564h16m5.451865679s) [noise=true , address=77.90.31.13:4627 , piece_id=3PDJWOWQ5776HPH2XHYWDBNSFX3CENHSBKXF4MTQYVAHI4MG2OMQ, seg_pos={0 0}, upload=496585]: Done uploading piece. err: <nil> Sep 22 02:24:30.110248880 (564h16m5.463034008s) [noise=true , address=45.91.116.4:4937 , piece_id=TWAKWSJ6THZF2P6E4EHSWOCNZCJGFJMYLCAWJ2UD6WTT63KBST5A, seg_pos={0 0}, upload=496585]: Done uploading piece. err: <nil> Sep 22 02:24:59.898601420 (564h16m35.251386518s) [noise=true , address=145.79.227.5:4223 , piece_id=G537GPGFD4JY7HEVFTQS64JZ5HFGBNSYIMNSN2ZVL6J2QQDE7XIA, seg_pos={0 0}, upload=496585]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: rpc: dial tcp 145.79.227.5:4223: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:24:59.898703411 (564h16m35.251488519s) [noise=true , address=145.79.227.11:4732 , piece_id=4BG4QVSKDMAU5WTQMHVQ3GMPFSHVW2CEIHHPV6R3BLQZA26KZLJA, seg_pos={0 0}, upload=496585]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:24:59.898720393 (564h16m35.251505501s) [noise=true , address=145.79.227.17:4971 , piece_id=KJA4H3RCIJFCLDYPF4VCSKBDGPC3RLDBVP37KRDEY44WA4B2YRNA, seg_pos={0 0}, upload=496585]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: rpc: dial tcp 145.79.227.17:4971: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:24:59.898746522 (564h16m35.25153163s) [noise=true , address=145.79.227.19:4078 , piece_id=O3SCR4MJDILV6JSGNN7CK4TIMQED4HEHFZMV5FFIWL7E5HD35R5A, seg_pos={0 0}, upload=496585]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:24:59.898769946 (564h16m35.251555054s) [seg_pos={0 0}, upload=496585]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:24:59.921079011 (564h16m35.273864119s) [upload=496585]: Done uploading segments... Sep 22 02:24:59.921097315 (564h16m35.273882413s) [upload=496585]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 551.847882ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [......................................X.XXXXXX..X...........] duration: 428.520819ms FAIL (2025-09-22 02:25:01 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== m30.954857157s) [seg_pos={0 0}, upload=496461]: Begin upload segment... Sep 22 02:15:55.602116061 (564h7m30.954901189s) [seg_pos={0 0}, upload=496461]: Done begin upload segment. Sep 22 02:15:55.602137471 (564h7m30.954922599s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602154613 (564h7m30.954939742s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602716609 (564h7m30.955501736s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602736295 (564h7m30.955521423s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603134873 (564h7m30.955920001s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603640462 (564h7m30.95642559s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604036745 (564h7m30.956821874s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604397323 (564h7m30.95718245s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604828131 (564h7m30.957613268s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605250674 (564h7m30.958035802s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605663548 (564h7m30.958448676s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606022221 (564h7m30.958807349s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606387297 (564h7m30.959172425s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606869060 (564h7m30.959654198s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607319626 (564h7m30.960104755s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607717662 (564h7m30.960502801s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608211139 (564h7m30.960996268s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608652857 (564h7m30.961437986s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.704831022 (564h7m31.05761613s) [upload=496461]: Next returned nil segment Sep 22 02:15:55.704846542 (564h7m31.05763165s) [upload=496461]: Waiting for error group managing segments... Sep 22 02:15:55.712709156 (564h7m31.065494285s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.713081926 (564h7m31.065867054s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.716351481 (564h7m31.06913661s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717069920 (564h7m31.069855048s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717303427 (564h7m31.070088556s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725190529 (564h7m31.077975657s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725273014 (564h7m31.078058142s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732220611 (564h7m31.085005749s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732337590 (564h7m31.085122719s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.734847580 (564h7m31.087632718s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.738226892 (564h7m31.09101203s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:16:55.564886089 (564h8m30.917671218s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1AowWwVvxzmnW5qwbqbMBVbvNoigffjAN51LQp8e6XA9CZkeXL, address:145.79.227.12:4790): protocol: rpc: dial tcp 145.79.227.12:4790: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564944549 (564h8m30.917729677s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564994583 (564h8m30.917779711s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565058042 (564h8m30.91784317s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1zywsXJnw5gfpUu7mC8p7UE7ev5HexPu5nSpUH1x2pzar99T97, address:145.79.227.8:4349): protocol: rpc: dial tcp 145.79.227.8:4349: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565089361 (564h8m30.917874489s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:145.79.227.14:4847): protocol: rpc: dial tcp 145.79.227.14:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565111382 (564h8m30.91789651s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12Pxos2xcjHwcYyQnXeRL3PL72G9pXaAnBs8XnkVFkxegDRPrqp, address:145.79.227.2:4127): protocol: rpc: dial tcp 145.79.227.2:4127: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565129266 (564h8m30.917914394s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12gs9LRE3oUp1jupCw5uDvRtkNcahbq2d4VTYQsdKr6atk6vGJx, address:145.79.227.19:4095): protocol: rpc: dial tcp 145.79.227.19:4095: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565148231 (564h8m30.917933359s) [seg_pos={0 0}, upload=496461]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:16:55.580701268 (564h8m30.933486366s) [upload=496461]: Done uploading segments... Sep 22 02:16:55.580714373 (564h8m30.93349947s) [upload=496461]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 428.542019ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [...................................XXX....XXX.XXXXX..X.XX.XX] duration: 30.024652224s FAIL (2025-09-22 02:24:59 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:24:29.913646711 (564h16m5.266431849s) [upload=496585]: Starting upload Sep 22 02:24:29.913754343 (564h16m5.266539481s) [upload=496585]: Done starting upload Sep 22 02:24:29.913826157 (564h16m5.266611296s) [upload=496585]: Uploading segments... Sep 22 02:24:29.914160415 (564h16m5.266945553s) [upload=496585]: Got next segment. Inline: false Sep 22 02:24:29.914231738 (564h16m5.267016866s) [upload=496585]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:24:29.914241637 (564h16m5.267026775s) [upload=496585]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:24:29.943512376 (564h16m5.296297504s) [seg_pos={0 0}, upload=496585]: Begin upload segment... Sep 22 02:24:29.943553513 (564h16m5.296338642s) [seg_pos={0 0}, upload=496585]: Done begin upload segment. Sep 22 02:24:29.943582378 (564h16m5.296367515s) [noise=true , address=45.91.116.9:4263 , piece_id=LPPLXP5T6REENEO7W2HT7ZSUI3ZHYEIPG37OHXORDF5B3Y6MGWUA, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.943612053 (564h16m5.296397182s) [noise=true , address=145.79.227.19:4078 , piece_id=O3SCR4MJDILV6JSGNN7CK4TIMQED4HEHFZMV5FFIWL7E5HD35R5A, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.944105910 (564h16m5.296891048s) [noise=true , address=77.90.31.21:4879 , piece_id=SOOFFNV3GFMFI7VXR5OJEHCGUHOQEP2XGS2SSSRTWDJDQV2QGFEA, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.944149352 (564h16m5.29693448s) [noise=true , address=145.79.227.11:4732 , piece_id=4BG4QVSKDMAU5WTQMHVQ3GMPFSHVW2CEIHHPV6R3BLQZA26KZLJA, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.944537811 (564h16m5.297322938s) [noise=true , address=77.90.31.11:4554 , piece_id=WAICDOYFSQTF2SYFYTZ2KPCISRSINU4HVJ24SVR5Q4QKOCY4QD2A, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.944550444 (564h16m5.297335583s) [noise=true , address=45.91.116.14:4190 , piece_id=3PZ6EV4HUPZGBMYI7M3X5C27KV4T6UNH62J2365D5RHPRQUUMZHA, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.944923614 (564h16m5.297708782s) [noise=true , address=45.91.116.4:4937 , piece_id=TWAKWSJ6THZF2P6E4EHSWOCNZCJGFJMYLCAWJ2UD6WTT63KBST5A, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.944966284 (564h16m5.297751422s) [noise=true , address=77.90.31.21:4877 , piece_id=BD6HJWSXDSDFBNRCQFYGSUYSFLDQDKAOT6AIFZBNM554YXIHUDAA, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.945367027 (564h16m5.298152165s) [noise=true , address=145.79.227.17:4971 , piece_id=KJA4H3RCIJFCLDYPF4VCSKBDGPC3RLDBVP37KRDEY44WA4B2YRNA, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.945403825 (564h16m5.298188954s) [noise=true , address=77.90.31.13:4627 , piece_id=3PDJWOWQ5776HPH2XHYWDBNSFX3CENHSBKXF4MTQYVAHI4MG2OMQ, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.945971571 (564h16m5.2987567s) [noise=true , address=145.79.227.5:4223 , piece_id=G537GPGFD4JY7HEVFTQS64JZ5HFGBNSYIMNSN2ZVL6J2QQDE7XIA, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.946404894 (564h16m5.299190023s) [noise=true , address=77.90.31.12:4589 , piece_id=DXJ7BQZETSXWHRX7K5D366ADUYJL5Q7EK3C65MCCN2KDD6XTNDMA, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.946622683 (564h16m5.299407811s) [noise=true , address=45.91.116.7:4010 , piece_id=BKVKLHYYXAPXUM2NDIAFUF7THYMNNFY4WSXMDTDN3NNXJ7XBQR4A, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.946886869 (564h16m5.299672048s) [noise=true , address=45.91.116.12:4226 , piece_id=MOD26FVJAHEO6CXLKDFCQH4FQWJXUGQGLU44WYIPHFJDYJPPFLGQ, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.947149081 (564h16m5.299934209s) [noise=true , address=77.90.31.18:4807 , piece_id=NT4UEGNET7FPI2HMLPDAFUIENECL6JG7LFA3ADPXQTP3KLGPMAQA, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:29.947312267 (564h16m5.300097395s) [noise=true , address=45.91.116.8:4047 , piece_id=WB7UZ72ATOKXWK3DGQ2J2PE3SE5Z3W77QR77GMPN6YMXK5KUDLBQ, seg_pos={0 0}, upload=496585]: Uploading piece... Sep 22 02:24:30.082454071 (564h16m5.435239189s) [upload=496585]: Next returned nil segment Sep 22 02:24:30.082468158 (564h16m5.435253276s) [upload=496585]: Waiting for error group managing segments... Sep 22 02:24:30.091223908 (564h16m5.444009016s) [noise=true , address=45.91.116.12:4226 , piece_id=MOD26FVJAHEO6CXLKDFCQH4FQWJXUGQGLU44WYIPHFJDYJPPFLGQ, seg_pos={0 0}, upload=496585]: Done uploading piece. err: <nil> Sep 22 02:24:30.091566551 (564h16m5.444351679s) [noise=true , address=45.91.116.7:4010 , piece_id=BKVKLHYYXAPXUM2NDIAFUF7THYMNNFY4WSXMDTDN3NNXJ7XBQR4A, seg_pos={0 0}, upload=496585]: Done uploading piece. err: <nil> Sep 22 02:24:30.091665817 (564h16m5.444450925s) [noise=true , address=77.90.31.21:4879 , piece_id=SOOFFNV3GFMFI7VXR5OJEHCGUHOQEP2XGS2SSSRTWDJDQV2QGFEA, seg_pos={0 0}, upload=496585]: Done uploading piece. err: <nil> Sep 22 02:24:30.092516043 (564h16m5.445301171s) [noise=true , address=45.91.116.9:4263 , piece_id=LPPLXP5T6REENEO7W2HT7ZSUI3ZHYEIPG37OHXORDF5B3Y6MGWUA, seg_pos={0 0}, upload=496585]: Done uploading piece. err: <nil> Sep 22 02:24:30.093580169 (564h16m5.446365288s) [noise=true , address=77.90.31.12:4589 , piece_id=DXJ7BQZETSXWHRX7K5D366ADUYJL5Q7EK3C65MCCN2KDD6XTNDMA, seg_pos={0 0}, upload=496585]: Done uploading piece. err: <nil> Sep 22 02:24:30.094634559 (564h16m5.447419677s) [noise=true , address=77.90.31.18:4807 , piece_id=NT4UEGNET7FPI2HMLPDAFUIENECL6JG7LFA3ADPXQTP3KLGPMAQA, seg_pos={0 0}, upload=496585]: Done uploading piece. err: <nil> Sep 22 02:24:30.095551219 (564h16m5.448336348s) [noise=true , address=77.90.31.11:4554 , piece_id=WAICDOYFSQTF2SYFYTZ2KPCISRSINU4HVJ24SVR5Q4QKOCY4QD2A, seg_pos={0 0}, upload=496585]: Done uploading piece. err: <nil> Sep 22 02:24:30.096503045 (564h16m5.449288174s) [noise=true , address=77.90.31.21:4877 , piece_id=BD6HJWSXDSDFBNRCQFYGSUYSFLDQDKAOT6AIFZBNM554YXIHUDAA, seg_pos={0 0}, upload=496585]: Done uploading piece. err: <nil> Sep 22 02:24:30.097369932 (564h16m5.450155059s) [noise=true , address=45.91.116.8:4047 , piece_id=WB7UZ72ATOKXWK3DGQ2J2PE3SE5Z3W77QR77GMPN6YMXK5KUDLBQ, seg_pos={0 0}, upload=496585]: Done uploading piece. err: <nil> Sep 22 02:24:30.097909815 (564h16m5.450694943s) [noise=true , address=45.91.116.14:4190 , piece_id=3PZ6EV4HUPZGBMYI7M3X5C27KV4T6UNH62J2365D5RHPRQUUMZHA, seg_pos={0 0}, upload=496585]: Done uploading piece. err: <nil> Sep 22 02:24:30.099080552 (564h16m5.451865679s) [noise=true , address=77.90.31.13:4627 , piece_id=3PDJWOWQ5776HPH2XHYWDBNSFX3CENHSBKXF4MTQYVAHI4MG2OMQ, seg_pos={0 0}, upload=496585]: Done uploading piece. err: <nil> Sep 22 02:24:30.110248880 (564h16m5.463034008s) [noise=true , address=45.91.116.4:4937 , piece_id=TWAKWSJ6THZF2P6E4EHSWOCNZCJGFJMYLCAWJ2UD6WTT63KBST5A, seg_pos={0 0}, upload=496585]: Done uploading piece. err: <nil> Sep 22 02:24:59.898601420 (564h16m35.251386518s) [noise=true , address=145.79.227.5:4223 , piece_id=G537GPGFD4JY7HEVFTQS64JZ5HFGBNSYIMNSN2ZVL6J2QQDE7XIA, seg_pos={0 0}, upload=496585]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: rpc: dial tcp 145.79.227.5:4223: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:24:59.898703411 (564h16m35.251488519s) [noise=true , address=145.79.227.11:4732 , piece_id=4BG4QVSKDMAU5WTQMHVQ3GMPFSHVW2CEIHHPV6R3BLQZA26KZLJA, seg_pos={0 0}, upload=496585]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:24:59.898720393 (564h16m35.251505501s) [noise=true , address=145.79.227.17:4971 , piece_id=KJA4H3RCIJFCLDYPF4VCSKBDGPC3RLDBVP37KRDEY44WA4B2YRNA, seg_pos={0 0}, upload=496585]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: rpc: dial tcp 145.79.227.17:4971: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:24:59.898746522 (564h16m35.25153163s) [noise=true , address=145.79.227.19:4078 , piece_id=O3SCR4MJDILV6JSGNN7CK4TIMQED4HEHFZMV5FFIWL7E5HD35R5A, seg_pos={0 0}, upload=496585]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:24:59.898769946 (564h16m35.251555054s) [seg_pos={0 0}, upload=496585]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:24:59.921079011 (564h16m35.273864119s) [upload=496585]: Done uploading segments... Sep 22 02:24:59.921097315 (564h16m35.273882413s) [upload=496585]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.024674596s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................................X.X.........X.......X] duration: 30.000806304s FAIL (2025-09-22 02:24:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000828346s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [.......................................X.XXXXXX..X..........] duration: 447.964359ms FAIL (2025-09-22 02:24:17 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== m30.954857157s) [seg_pos={0 0}, upload=496461]: Begin upload segment... Sep 22 02:15:55.602116061 (564h7m30.954901189s) [seg_pos={0 0}, upload=496461]: Done begin upload segment. Sep 22 02:15:55.602137471 (564h7m30.954922599s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602154613 (564h7m30.954939742s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602716609 (564h7m30.955501736s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602736295 (564h7m30.955521423s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603134873 (564h7m30.955920001s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603640462 (564h7m30.95642559s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604036745 (564h7m30.956821874s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604397323 (564h7m30.95718245s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604828131 (564h7m30.957613268s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605250674 (564h7m30.958035802s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605663548 (564h7m30.958448676s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606022221 (564h7m30.958807349s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606387297 (564h7m30.959172425s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606869060 (564h7m30.959654198s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607319626 (564h7m30.960104755s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607717662 (564h7m30.960502801s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608211139 (564h7m30.960996268s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608652857 (564h7m30.961437986s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.704831022 (564h7m31.05761613s) [upload=496461]: Next returned nil segment Sep 22 02:15:55.704846542 (564h7m31.05763165s) [upload=496461]: Waiting for error group managing segments... Sep 22 02:15:55.712709156 (564h7m31.065494285s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.713081926 (564h7m31.065867054s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.716351481 (564h7m31.06913661s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717069920 (564h7m31.069855048s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717303427 (564h7m31.070088556s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725190529 (564h7m31.077975657s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725273014 (564h7m31.078058142s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732220611 (564h7m31.085005749s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732337590 (564h7m31.085122719s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.734847580 (564h7m31.087632718s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.738226892 (564h7m31.09101203s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:16:55.564886089 (564h8m30.917671218s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1AowWwVvxzmnW5qwbqbMBVbvNoigffjAN51LQp8e6XA9CZkeXL, address:145.79.227.12:4790): protocol: rpc: dial tcp 145.79.227.12:4790: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564944549 (564h8m30.917729677s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564994583 (564h8m30.917779711s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565058042 (564h8m30.91784317s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1zywsXJnw5gfpUu7mC8p7UE7ev5HexPu5nSpUH1x2pzar99T97, address:145.79.227.8:4349): protocol: rpc: dial tcp 145.79.227.8:4349: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565089361 (564h8m30.917874489s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:145.79.227.14:4847): protocol: rpc: dial tcp 145.79.227.14:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565111382 (564h8m30.91789651s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12Pxos2xcjHwcYyQnXeRL3PL72G9pXaAnBs8XnkVFkxegDRPrqp, address:145.79.227.2:4127): protocol: rpc: dial tcp 145.79.227.2:4127: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565129266 (564h8m30.917914394s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12gs9LRE3oUp1jupCw5uDvRtkNcahbq2d4VTYQsdKr6atk6vGJx, address:145.79.227.19:4095): protocol: rpc: dial tcp 145.79.227.19:4095: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565148231 (564h8m30.917933359s) [seg_pos={0 0}, upload=496461]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:16:55.580701268 (564h8m30.933486366s) [upload=496461]: Done uploading segments... Sep 22 02:16:55.580714373 (564h8m30.93349947s) [upload=496461]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 447.994455ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [....................................XXX....XXX.XXXXX..X.XX.X] duration: 30.016072286s FAIL (2025-09-22 02:24:16 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:23:46.775604078 (564h15m22.128389206s) [upload=496573]: Starting upload Sep 22 02:23:46.775681814 (564h15m22.128466942s) [upload=496573]: Done starting upload Sep 22 02:23:46.775838808 (564h15m22.128623937s) [upload=496573]: Uploading segments... Sep 22 02:23:46.776098326 (564h15m22.128883454s) [upload=496573]: Got next segment. Inline: false Sep 22 02:23:46.776145635 (564h15m22.128930763s) [upload=496573]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:23:46.776151015 (564h15m22.128936143s) [upload=496573]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:23:46.800657412 (564h15m22.15344254s) [seg_pos={0 0}, upload=496573]: Begin upload segment... Sep 22 02:23:46.800692007 (564h15m22.153477135s) [seg_pos={0 0}, upload=496573]: Done begin upload segment. Sep 22 02:23:46.800724289 (564h15m22.153509416s) [noise=true , address=77.90.31.15:4697 , piece_id=RRB7PMF53TZAVSE3RLVVTLPFSLEW4U7FM5HT4TYHWQTDRBCYYTNA, seg_pos={0 0}, upload=496573]: Uploading piece... Sep 22 02:23:46.800774362 (564h15m22.153559491s) [noise=true , address=45.91.116.8:4048 , piece_id=ZF747DVM63HSIJYGX3AAFVMT2VYYREW5UUQNQEEAZHAW3G6RYMZQ, seg_pos={0 0}, upload=496573]: Uploading piece... Sep 22 02:23:46.801180104 (564h15m22.153965241s) [noise=true , address=145.79.227.15:4873 , piece_id=OG64SBE73MOC3CWGPOAIXLSICPVWA7SN7MC2DUXFFTRXWKQBLHWA, seg_pos={0 0}, upload=496573]: Uploading piece... Sep 22 02:23:46.801219717 (564h15m22.154004846s) [noise=true , address=77.90.31.10:4519 , piece_id=7VHAVXUYJ77CXPNO3S3QGNIH7DIXOKWNXSDUJGVSGUMX5ELMAERA, seg_pos={0 0}, upload=496573]: Uploading piece... Sep 22 02:23:46.801575615 (564h15m22.154360743s) [noise=true , address=77.90.31.12:4590 , piece_id=KTFTQKXOGFO66W67XCQOP6U77WKBQE72UGZSMDTCHLN2HCP5I6WQ, seg_pos={0 0}, upload=496573]: Uploading piece... Sep 22 02:23:46.801617624 (564h15m22.154402752s) [noise=true , address=77.90.31.18:4805 , piece_id=DUDJWBKALJAKPB42SOMV7M34VUOWWTTKZYAX6VRDEZZGKBYI6AFQ, seg_pos={0 0}, upload=496573]: Uploading piece... Sep 22 02:23:46.801939068 (564h15m22.154724206s) [noise=true , address=45.91.116.5:4298 , piece_id=EAEQK2KDCFQO4QNNIEIP6VP7YUYG7PTMAWWF7XJDMKY6BAMERTJA, seg_pos={0 0}, upload=496573]: Uploading piece... Sep 22 02:23:46.801992188 (564h15m22.154777325s) [noise=true , address=145.79.227.18:4006 , piece_id=VHEHCA7VSQP5Z4RWN7UCDV7I4TZLY5KBZ5VMBRXXYEUWO4RZHLHA, seg_pos={0 0}, upload=496573]: Uploading piece... Sep 22 02:23:46.802287611 (564h15m22.15507274s) [noise=true , address=45.91.116.5:4299 , piece_id=BJVWQJXVV5FZO732STEZUQZOR553WGGR2ZITQXYQX2FUZNK4WODA, seg_pos={0 0}, upload=496573]: Uploading piece... Sep 22 02:23:46.802496413 (564h15m22.155281541s) [noise=true , address=77.90.31.13:4627 , piece_id=XX44GH4JZCQX4VFTCMOPTXHE2AX3GBF3GCQAESC6FNSRL3JABRKA, seg_pos={0 0}, upload=496573]: Uploading piece... Sep 22 02:23:46.802685088 (564h15m22.155470226s) [noise=true , address=45.91.116.10:4081 , piece_id=463F6Z5OKKDMUUDSBPCXOCINI5CDNIAZTCBLQPBJD5WQJ7A5PKCA, seg_pos={0 0}, upload=496573]: Uploading piece... Sep 22 02:23:46.802911021 (564h15m22.15569615s) [noise=true , address=45.91.116.4:4938 , piece_id=ZYJU7X3JPH5JL6H4TNGFCVLIGD2NGS3MFL5RFYWKDBE4IGSWQUIQ, seg_pos={0 0}, upload=496573]: Uploading piece... Sep 22 02:23:46.803043891 (564h15m22.155829029s) [noise=true , address=145.79.227.11:4732 , piece_id=6YPOQEGKA4WFAHTN5O3ZA77WZKRDOZ67UTFS4QGGRCGH5PTBK4CQ, seg_pos={0 0}, upload=496573]: Uploading piece... Sep 22 02:23:46.803279102 (564h15m22.15606423s) [noise=true , address=45.91.116.14:4190 , piece_id=NGID6MZSYQRZHFCDMK4FRL3H6UO5ZCRIXWD526HUPJ7XXULTV6OQ, seg_pos={0 0}, upload=496573]: Uploading piece... Sep 22 02:23:46.803391833 (564h15m22.156176972s) [noise=true , address=145.79.227.12:4767 , piece_id=QMZXF3Q3RYOAMQBXPCPR7CPSDW3PJBT2CERJFTAUPPMOAIYUBHYA, seg_pos={0 0}, upload=496573]: Uploading piece... Sep 22 02:23:46.803771155 (564h15m22.156556284s) [noise=true , address=145.79.227.14:4837 , piece_id=TCFGR2RDKB2EQDPZPGUOCRMIEYKTYVGHRIELD26JWXTRTU4I4CLQ, seg_pos={0 0}, upload=496573]: Uploading piece... Sep 22 02:23:46.914599592 (564h15m22.26738472s) [upload=496573]: Next returned nil segment Sep 22 02:23:46.914614020 (564h15m22.267399158s) [upload=496573]: Waiting for error group managing segments... Sep 22 02:23:46.923431607 (564h15m22.276216735s) [noise=true , address=45.91.116.14:4190 , piece_id=NGID6MZSYQRZHFCDMK4FRL3H6UO5ZCRIXWD526HUPJ7XXULTV6OQ, seg_pos={0 0}, upload=496573]: Done uploading piece. err: <nil> Sep 22 02:23:46.924106002 (564h15m22.27689113s) [noise=true , address=45.91.116.4:4938 , piece_id=ZYJU7X3JPH5JL6H4TNGFCVLIGD2NGS3MFL5RFYWKDBE4IGSWQUIQ, seg_pos={0 0}, upload=496573]: Done uploading piece. err: <nil> Sep 22 02:23:46.924706078 (564h15m22.277491206s) [noise=true , address=45.91.116.8:4048 , piece_id=ZF747DVM63HSIJYGX3AAFVMT2VYYREW5UUQNQEEAZHAW3G6RYMZQ, seg_pos={0 0}, upload=496573]: Done uploading piece. err: <nil> Sep 22 02:23:46.925264607 (564h15m22.278049735s) [noise=true , address=45.91.116.5:4298 , piece_id=EAEQK2KDCFQO4QNNIEIP6VP7YUYG7PTMAWWF7XJDMKY6BAMERTJA, seg_pos={0 0}, upload=496573]: Done uploading piece. err: <nil> Sep 22 02:23:46.925310663 (564h15m22.278095792s) [noise=true , address=45.91.116.10:4081 , piece_id=463F6Z5OKKDMUUDSBPCXOCINI5CDNIAZTCBLQPBJD5WQJ7A5PKCA, seg_pos={0 0}, upload=496573]: Done uploading piece. err: <nil> Sep 22 02:23:46.925538040 (564h15m22.278323167s) [noise=true , address=45.91.116.5:4299 , piece_id=BJVWQJXVV5FZO732STEZUQZOR553WGGR2ZITQXYQX2FUZNK4WODA, seg_pos={0 0}, upload=496573]: Done uploading piece. err: <nil> Sep 22 02:23:46.925992843 (564h15m22.278777971s) [noise=true , address=77.90.31.15:4697 , piece_id=RRB7PMF53TZAVSE3RLVVTLPFSLEW4U7FM5HT4TYHWQTDRBCYYTNA, seg_pos={0 0}, upload=496573]: Done uploading piece. err: <nil> Sep 22 02:23:46.926604481 (564h15m22.279389608s) [noise=true , address=77.90.31.18:4805 , piece_id=DUDJWBKALJAKPB42SOMV7M34VUOWWTTKZYAX6VRDEZZGKBYI6AFQ, seg_pos={0 0}, upload=496573]: Done uploading piece. err: <nil> Sep 22 02:23:46.926762186 (564h15m22.279547315s) [noise=true , address=77.90.31.13:4627 , piece_id=XX44GH4JZCQX4VFTCMOPTXHE2AX3GBF3GCQAESC6FNSRL3JABRKA, seg_pos={0 0}, upload=496573]: Done uploading piece. err: <nil> Sep 22 02:23:46.927156708 (564h15m22.279941835s) [noise=true , address=77.90.31.12:4590 , piece_id=KTFTQKXOGFO66W67XCQOP6U77WKBQE72UGZSMDTCHLN2HCP5I6WQ, seg_pos={0 0}, upload=496573]: Done uploading piece. err: <nil> Sep 22 02:23:46.927590461 (564h15m22.280375599s) [noise=true , address=77.90.31.10:4519 , piece_id=7VHAVXUYJ77CXPNO3S3QGNIH7DIXOKWNXSDUJGVSGUMX5ELMAERA, seg_pos={0 0}, upload=496573]: Done uploading piece. err: <nil> Sep 22 02:24:16.760683382 (564h15m52.11346848s) [noise=true , address=145.79.227.15:4873 , piece_id=OG64SBE73MOC3CWGPOAIXLSICPVWA7SN7MC2DUXFFTRXWKQBLHWA, seg_pos={0 0}, upload=496573]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:24:16.760776207 (564h15m52.113561315s) [noise=true , address=145.79.227.18:4006 , piece_id=VHEHCA7VSQP5Z4RWN7UCDV7I4TZLY5KBZ5VMBRXXYEUWO4RZHLHA, seg_pos={0 0}, upload=496573]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:24:16.760787919 (564h15m52.113573027s) [noise=true , address=145.79.227.11:4732 , piece_id=6YPOQEGKA4WFAHTN5O3ZA77WZKRDOZ67UTFS4QGGRCGH5PTBK4CQ, seg_pos={0 0}, upload=496573]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:24:16.760861668 (564h15m52.113646776s) [noise=true , address=145.79.227.14:4837 , piece_id=TCFGR2RDKB2EQDPZPGUOCRMIEYKTYVGHRIELD26JWXTRTU4I4CLQ, seg_pos={0 0}, upload=496573]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:24:16.760872067 (564h15m52.113657175s) [noise=true , address=145.79.227.12:4767 , piece_id=QMZXF3Q3RYOAMQBXPCPR7CPSDW3PJBT2CERJFTAUPPMOAIYUBHYA, seg_pos={0 0}, upload=496573]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:24:16.760892114 (564h15m52.113677252s) [seg_pos={0 0}, upload=496573]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:24:16.775204162 (564h15m52.12798926s) [upload=496573]: Done uploading segments... Sep 22 02:24:16.775214311 (564h15m52.127999419s) [upload=496573]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.016092393s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [........................................X.XXXXXX..X.........] duration: 449.289954ms FAIL (2025-09-22 02:23:13 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== m30.954857157s) [seg_pos={0 0}, upload=496461]: Begin upload segment... Sep 22 02:15:55.602116061 (564h7m30.954901189s) [seg_pos={0 0}, upload=496461]: Done begin upload segment. Sep 22 02:15:55.602137471 (564h7m30.954922599s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602154613 (564h7m30.954939742s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602716609 (564h7m30.955501736s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602736295 (564h7m30.955521423s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603134873 (564h7m30.955920001s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603640462 (564h7m30.95642559s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604036745 (564h7m30.956821874s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604397323 (564h7m30.95718245s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604828131 (564h7m30.957613268s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605250674 (564h7m30.958035802s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605663548 (564h7m30.958448676s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606022221 (564h7m30.958807349s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606387297 (564h7m30.959172425s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606869060 (564h7m30.959654198s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607319626 (564h7m30.960104755s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607717662 (564h7m30.960502801s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608211139 (564h7m30.960996268s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608652857 (564h7m30.961437986s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.704831022 (564h7m31.05761613s) [upload=496461]: Next returned nil segment Sep 22 02:15:55.704846542 (564h7m31.05763165s) [upload=496461]: Waiting for error group managing segments... Sep 22 02:15:55.712709156 (564h7m31.065494285s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.713081926 (564h7m31.065867054s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.716351481 (564h7m31.06913661s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717069920 (564h7m31.069855048s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717303427 (564h7m31.070088556s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725190529 (564h7m31.077975657s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725273014 (564h7m31.078058142s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732220611 (564h7m31.085005749s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732337590 (564h7m31.085122719s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.734847580 (564h7m31.087632718s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.738226892 (564h7m31.09101203s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:16:55.564886089 (564h8m30.917671218s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1AowWwVvxzmnW5qwbqbMBVbvNoigffjAN51LQp8e6XA9CZkeXL, address:145.79.227.12:4790): protocol: rpc: dial tcp 145.79.227.12:4790: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564944549 (564h8m30.917729677s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564994583 (564h8m30.917779711s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565058042 (564h8m30.91784317s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1zywsXJnw5gfpUu7mC8p7UE7ev5HexPu5nSpUH1x2pzar99T97, address:145.79.227.8:4349): protocol: rpc: dial tcp 145.79.227.8:4349: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565089361 (564h8m30.917874489s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:145.79.227.14:4847): protocol: rpc: dial tcp 145.79.227.14:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565111382 (564h8m30.91789651s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12Pxos2xcjHwcYyQnXeRL3PL72G9pXaAnBs8XnkVFkxegDRPrqp, address:145.79.227.2:4127): protocol: rpc: dial tcp 145.79.227.2:4127: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565129266 (564h8m30.917914394s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12gs9LRE3oUp1jupCw5uDvRtkNcahbq2d4VTYQsdKr6atk6vGJx, address:145.79.227.19:4095): protocol: rpc: dial tcp 145.79.227.19:4095: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565148231 (564h8m30.917933359s) [seg_pos={0 0}, upload=496461]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:16:55.580701268 (564h8m30.933486366s) [upload=496461]: Done uploading segments... Sep 22 02:16:55.580714373 (564h8m30.93349947s) [upload=496461]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 449.319379ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................................X.X.........X.......] duration: 444.770496ms FAIL (2025-09-22 02:23:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 444.793941ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [.....................................XXX....XXX.XXXXX..X.XX.] duration: 453.295069ms FAIL (2025-09-22 02:22:57 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Sep 22 02:22:09.070164417 (564h13m44.422949555s) [upload=496551]: Starting upload Sep 22 02:22:09.070269996 (564h13m44.423055134s) [upload=496551]: Done starting upload Sep 22 02:22:09.070334978 (564h13m44.423120106s) [upload=496551]: Uploading segments... Sep 22 02:22:09.070719549 (564h13m44.423504688s) [upload=496551]: Got next segment. Inline: false Sep 22 02:22:09.070773761 (564h13m44.423558899s) [upload=496551]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:22:09.070781656 (564h13m44.423566794s) [upload=496551]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:22:09.180192945 (564h13m44.532978083s) [seg_pos={0 0}, upload=496551]: Begin upload segment... Sep 22 02:22:09.180247357 (564h13m44.533032485s) [seg_pos={0 0}, upload=496551]: Done begin upload segment. Sep 22 02:22:09.180276712 (564h13m44.53306184s) [noise=true , address=145.79.227.6:4258 , piece_id=BQPKF37ODCA56JJ2RHLPNBSWMADXOWZRR6TGHVCY4ZB4TPCZC77A, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.180301287 (564h13m44.533086416s) [noise=true , address=77.90.31.21:4878 , piece_id=4RFPKSZEZPH3Y7QIQFQMF24EABH52FEUYCNBDIWHG5G4NC2ZZBQQ, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.180840670 (564h13m44.533625807s) [noise=true , address=77.90.31.12:4590 , piece_id=3GANTEOT4LO65G4PDSS6DVVCPYOERTZZGGDPZDA6EB376UYMRVVQ, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.180849286 (564h13m44.533634414s) [noise=true , address=45.91.116.14:4191 , piece_id=URYYG4NYG6JI754Y37PT27MNYF3YCQHNIFKQEW3I7WDCMNL3QUIA, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.181233257 (564h13m44.534018385s) [noise=true , address=45.91.116.9:4261 , piece_id=37PHHPAWDXLBPE56XVH46QPXTXNSX7GRXMSPVANWJUBV5CSPYEPQ, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.181242144 (564h13m44.534027271s) [noise=true , address=45.91.116.5:4298 , piece_id=OGPIA3MYBZS2RKRI7G6YQUNUUDIKJ52DIERC5KRZRZFGTOUILKQA, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.181630141 (564h13m44.534415269s) [noise=true , address=145.79.227.19:4077 , piece_id=FW6272PRMJWOZWVWPTNEK3HCGIODO7YQ2OPFYZTDZ52UAR6MKUWA, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.181681488 (564h13m44.534466615s) [noise=true , address=145.79.227.5:4223 , piece_id=BQJUF4L47G5MNFL3OMGH2AZXLFTBG52VDLIDWZXSMTGW4D5OAENA, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.182001769 (564h13m44.534786897s) [noise=true , address=145.79.227.13:4802 , piece_id=LTHSKOSHKC2YYCJE5XO5XII2WQZA5UEONG5WVXXEB5J33PUGOHOA, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.182043958 (564h13m44.534829086s) [noise=true , address=45.91.116.6:4975 , piece_id=PBXYHGT3I4P7ZJ3HASGR5IHI5NVFGA7R5VYJBZ7FBXEB7BKXKCKA, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.182375310 (564h13m44.535160438s) [noise=true , address=145.79.227.2:4113 , piece_id=LPHCGCOYFXQEBKMPNG3BXK67FJNCRI27C6YQCRJ5VJCOOIXLLZ6Q, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.182385529 (564h13m44.535170667s) [noise=true , address=145.79.227.11:4731 , piece_id=VZWBU4ZXL6M5KPQDET7RGZR5BS5GB2UJXMJFXQUKT6Z5OCKE3KJQ, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.182741777 (564h13m44.535526895s) [noise=true , address=45.91.116.4:4938 , piece_id=R3NOVEHFPRHZ73CDORFOCCHTOOYFIJWFVJJYYQ4VKDOB6OQPFHRA, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.182795187 (564h13m44.535580326s) [noise=true , address=45.91.116.11:4117 , piece_id=RFKTUYY272OABR56VGDXBG7YKTAAIBTMP5IUBV6YMMVOIHIBPZCA, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.183078529 (564h13m44.535863658s) [noise=true , address=45.91.116.8:4045 , piece_id=CYZ5LMTPWHDFGWPNAHTD4NCBGIIIY76LS2D2EXH7TQ4NHFWEOH4Q, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.183177015 (564h13m44.535962142s) [noise=true , address=145.79.227.4:4187 , piece_id=RAN3UTPCEQ3523F5ZCEVF64CWKK7RJWMUD5OVVLF6IBGF7TJK7AA, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.288931400 (564h13m44.641716548s) [upload=496551]: Next returned nil segment Sep 22 02:22:09.288953130 (564h13m44.641738309s) [upload=496551]: Waiting for error group managing segments... Sep 22 02:22:09.297528322 (564h13m44.65031345s) [noise=true , address=45.91.116.9:4261 , piece_id=37PHHPAWDXLBPE56XVH46QPXTXNSX7GRXMSPVANWJUBV5CSPYEPQ, seg_pos={0 0}, upload=496551]: Done uploading piece. err: <nil> Sep 22 02:22:09.298240008 (564h13m44.651025146s) [noise=true , address=45.91.116.6:4975 , piece_id=PBXYHGT3I4P7ZJ3HASGR5IHI5NVFGA7R5VYJBZ7FBXEB7BKXKCKA, seg_pos={0 0}, upload=496551]: Done uploading piece. err: <nil> Sep 22 02:22:09.298321902 (564h13m44.651107029s) [noise=true , address=45.91.116.14:4191 , piece_id=URYYG4NYG6JI754Y37PT27MNYF3YCQHNIFKQEW3I7WDCMNL3QUIA, seg_pos={0 0}, upload=496551]: Done uploading piece. err: <nil> Sep 22 02:22:09.298893364 (564h13m44.651678492s) [noise=true , address=77.90.31.12:4590 , piece_id=3GANTEOT4LO65G4PDSS6DVVCPYOERTZZGGDPZDA6EB376UYMRVVQ, seg_pos={0 0}, upload=496551]: Done uploading piece. err: <nil> Sep 22 02:22:09.299154804 (564h13m44.651939933s) [noise=true , address=45.91.116.4:4938 , piece_id=R3NOVEHFPRHZ73CDORFOCCHTOOYFIJWFVJJYYQ4VKDOB6OQPFHRA, seg_pos={0 0}, upload=496551]: Done uploading piece. err: <nil> Sep 22 02:22:09.299479363 (564h13m44.652264501s) [noise=true , address=45.91.116.11:4117 , piece_id=RFKTUYY272OABR56VGDXBG7YKTAAIBTMP5IUBV6YMMVOIHIBPZCA, seg_pos={0 0}, upload=496551]: Done uploading piece. err: <nil> Sep 22 02:22:09.299893821 (564h13m44.65267895s) [noise=true , address=45.91.116.8:4045 , piece_id=CYZ5LMTPWHDFGWPNAHTD4NCBGIIIY76LS2D2EXH7TQ4NHFWEOH4Q, seg_pos={0 0}, upload=496551]: Done uploading piece. err: <nil> Sep 22 02:22:09.301609902 (564h13m44.65439503s) [noise=true , address=77.90.31.21:4878 , piece_id=4RFPKSZEZPH3Y7QIQFQMF24EABH52FEUYCNBDIWHG5G4NC2ZZBQQ, seg_pos={0 0}, upload=496551]: Done uploading piece. err: <nil> Sep 22 02:22:09.308864345 (564h13m44.661649483s) [noise=true , address=45.91.116.5:4298 , piece_id=OGPIA3MYBZS2RKRI7G6YQUNUUDIKJ52DIERC5KRZRZFGTOUILKQA, seg_pos={0 0}, upload=496551]: Done uploading piece. err: <nil> Sep 22 02:22:39.055394557 (564h14m14.408179685s) [noise=true , address=145.79.227.13:4802 , piece_id=LTHSKOSHKC2YYCJE5XO5XII2WQZA5UEONG5WVXXEB5J33PUGOHOA, seg_pos={0 0}, upload=496551]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:22:39.055501067 (564h14m14.408286195s) [noise=true , address=145.79.227.2:4113 , piece_id=LPHCGCOYFXQEBKMPNG3BXK67FJNCRI27C6YQCRJ5VJCOOIXLLZ6Q, seg_pos={0 0}, upload=496551]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: rpc: dial tcp 145.79.227.2:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:22:39.055676616 (564h14m14.408461744s) [noise=true , address=145.79.227.5:4223 , piece_id=BQJUF4L47G5MNFL3OMGH2AZXLFTBG52VDLIDWZXSMTGW4D5OAENA, seg_pos={0 0}, upload=496551]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: rpc: dial tcp 145.79.227.5:4223: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:22:39.055725277 (564h14m14.408510405s) [noise=true , address=145.79.227.11:4731 , piece_id=VZWBU4ZXL6M5KPQDET7RGZR5BS5GB2UJXMJFXQUKT6Z5OCKE3KJQ, seg_pos={0 0}, upload=496551]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:22:39.055746828 (564h14m14.408531956s) [noise=true , address=145.79.227.19:4077 , piece_id=FW6272PRMJWOZWVWPTNEK3HCGIODO7YQ2OPFYZTDZ52UAR6MKUWA, seg_pos={0 0}, upload=496551]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:145.79.227.19:4077): protocol: rpc: dial tcp 145.79.227.19:4077: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:22:39.055805578 (564h14m14.408590706s) [noise=true , address=145.79.227.4:4187 , piece_id=RAN3UTPCEQ3523F5ZCEVF64CWKK7RJWMUD5OVVLF6IBGF7TJK7AA, seg_pos={0 0}, upload=496551]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:22:39.055819244 (564h14m14.408604381s) [noise=true , address=145.79.227.6:4258 , piece_id=BQPKF37ODCA56JJ2RHLPNBSWMADXOWZRR6TGHVCY4ZB4TPCZC77A, seg_pos={0 0}, upload=496551]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: rpc: dial tcp 145.79.227.6:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:22:39.055864699 (564h14m14.408649826s) [seg_pos={0 0}, upload=496551]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 22 02:22:39.070591375 (564h14m14.423376502s) [upload=496551]: Done uploading segments... Sep 22 02:22:39.070607855 (564h14m14.423392984s) [upload=496551]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 453.324997ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [.........................................X.XXXXXX..X........] duration: 733.593622ms FAIL (2025-09-22 02:22:54 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== m30.954857157s) [seg_pos={0 0}, upload=496461]: Begin upload segment... Sep 22 02:15:55.602116061 (564h7m30.954901189s) [seg_pos={0 0}, upload=496461]: Done begin upload segment. Sep 22 02:15:55.602137471 (564h7m30.954922599s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602154613 (564h7m30.954939742s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602716609 (564h7m30.955501736s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602736295 (564h7m30.955521423s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603134873 (564h7m30.955920001s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603640462 (564h7m30.95642559s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604036745 (564h7m30.956821874s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604397323 (564h7m30.95718245s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604828131 (564h7m30.957613268s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605250674 (564h7m30.958035802s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605663548 (564h7m30.958448676s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606022221 (564h7m30.958807349s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606387297 (564h7m30.959172425s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606869060 (564h7m30.959654198s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607319626 (564h7m30.960104755s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607717662 (564h7m30.960502801s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608211139 (564h7m30.960996268s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608652857 (564h7m30.961437986s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.704831022 (564h7m31.05761613s) [upload=496461]: Next returned nil segment Sep 22 02:15:55.704846542 (564h7m31.05763165s) [upload=496461]: Waiting for error group managing segments... Sep 22 02:15:55.712709156 (564h7m31.065494285s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.713081926 (564h7m31.065867054s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.716351481 (564h7m31.06913661s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717069920 (564h7m31.069855048s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717303427 (564h7m31.070088556s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725190529 (564h7m31.077975657s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725273014 (564h7m31.078058142s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732220611 (564h7m31.085005749s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732337590 (564h7m31.085122719s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.734847580 (564h7m31.087632718s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.738226892 (564h7m31.09101203s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:16:55.564886089 (564h8m30.917671218s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1AowWwVvxzmnW5qwbqbMBVbvNoigffjAN51LQp8e6XA9CZkeXL, address:145.79.227.12:4790): protocol: rpc: dial tcp 145.79.227.12:4790: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564944549 (564h8m30.917729677s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564994583 (564h8m30.917779711s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565058042 (564h8m30.91784317s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1zywsXJnw5gfpUu7mC8p7UE7ev5HexPu5nSpUH1x2pzar99T97, address:145.79.227.8:4349): protocol: rpc: dial tcp 145.79.227.8:4349: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565089361 (564h8m30.917874489s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:145.79.227.14:4847): protocol: rpc: dial tcp 145.79.227.14:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565111382 (564h8m30.91789651s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12Pxos2xcjHwcYyQnXeRL3PL72G9pXaAnBs8XnkVFkxegDRPrqp, address:145.79.227.2:4127): protocol: rpc: dial tcp 145.79.227.2:4127: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565129266 (564h8m30.917914394s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12gs9LRE3oUp1jupCw5uDvRtkNcahbq2d4VTYQsdKr6atk6vGJx, address:145.79.227.19:4095): protocol: rpc: dial tcp 145.79.227.19:4095: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565148231 (564h8m30.917933359s) [seg_pos={0 0}, upload=496461]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:16:55.580701268 (564h8m30.933486366s) [upload=496461]: Done uploading segments... Sep 22 02:16:55.580714373 (564h8m30.93349947s) [upload=496461]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 733.618988ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [......................................XXX....XXX.XXXXX..X.XX] duration: 30.016843294s FAIL (2025-09-22 02:22:39 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Sep 22 02:22:09.070164417 (564h13m44.422949555s) [upload=496551]: Starting upload Sep 22 02:22:09.070269996 (564h13m44.423055134s) [upload=496551]: Done starting upload Sep 22 02:22:09.070334978 (564h13m44.423120106s) [upload=496551]: Uploading segments... Sep 22 02:22:09.070719549 (564h13m44.423504688s) [upload=496551]: Got next segment. Inline: false Sep 22 02:22:09.070773761 (564h13m44.423558899s) [upload=496551]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:22:09.070781656 (564h13m44.423566794s) [upload=496551]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:22:09.180192945 (564h13m44.532978083s) [seg_pos={0 0}, upload=496551]: Begin upload segment... Sep 22 02:22:09.180247357 (564h13m44.533032485s) [seg_pos={0 0}, upload=496551]: Done begin upload segment. Sep 22 02:22:09.180276712 (564h13m44.53306184s) [noise=true , address=145.79.227.6:4258 , piece_id=BQPKF37ODCA56JJ2RHLPNBSWMADXOWZRR6TGHVCY4ZB4TPCZC77A, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.180301287 (564h13m44.533086416s) [noise=true , address=77.90.31.21:4878 , piece_id=4RFPKSZEZPH3Y7QIQFQMF24EABH52FEUYCNBDIWHG5G4NC2ZZBQQ, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.180840670 (564h13m44.533625807s) [noise=true , address=77.90.31.12:4590 , piece_id=3GANTEOT4LO65G4PDSS6DVVCPYOERTZZGGDPZDA6EB376UYMRVVQ, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.180849286 (564h13m44.533634414s) [noise=true , address=45.91.116.14:4191 , piece_id=URYYG4NYG6JI754Y37PT27MNYF3YCQHNIFKQEW3I7WDCMNL3QUIA, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.181233257 (564h13m44.534018385s) [noise=true , address=45.91.116.9:4261 , piece_id=37PHHPAWDXLBPE56XVH46QPXTXNSX7GRXMSPVANWJUBV5CSPYEPQ, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.181242144 (564h13m44.534027271s) [noise=true , address=45.91.116.5:4298 , piece_id=OGPIA3MYBZS2RKRI7G6YQUNUUDIKJ52DIERC5KRZRZFGTOUILKQA, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.181630141 (564h13m44.534415269s) [noise=true , address=145.79.227.19:4077 , piece_id=FW6272PRMJWOZWVWPTNEK3HCGIODO7YQ2OPFYZTDZ52UAR6MKUWA, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.181681488 (564h13m44.534466615s) [noise=true , address=145.79.227.5:4223 , piece_id=BQJUF4L47G5MNFL3OMGH2AZXLFTBG52VDLIDWZXSMTGW4D5OAENA, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.182001769 (564h13m44.534786897s) [noise=true , address=145.79.227.13:4802 , piece_id=LTHSKOSHKC2YYCJE5XO5XII2WQZA5UEONG5WVXXEB5J33PUGOHOA, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.182043958 (564h13m44.534829086s) [noise=true , address=45.91.116.6:4975 , piece_id=PBXYHGT3I4P7ZJ3HASGR5IHI5NVFGA7R5VYJBZ7FBXEB7BKXKCKA, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.182375310 (564h13m44.535160438s) [noise=true , address=145.79.227.2:4113 , piece_id=LPHCGCOYFXQEBKMPNG3BXK67FJNCRI27C6YQCRJ5VJCOOIXLLZ6Q, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.182385529 (564h13m44.535170667s) [noise=true , address=145.79.227.11:4731 , piece_id=VZWBU4ZXL6M5KPQDET7RGZR5BS5GB2UJXMJFXQUKT6Z5OCKE3KJQ, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.182741777 (564h13m44.535526895s) [noise=true , address=45.91.116.4:4938 , piece_id=R3NOVEHFPRHZ73CDORFOCCHTOOYFIJWFVJJYYQ4VKDOB6OQPFHRA, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.182795187 (564h13m44.535580326s) [noise=true , address=45.91.116.11:4117 , piece_id=RFKTUYY272OABR56VGDXBG7YKTAAIBTMP5IUBV6YMMVOIHIBPZCA, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.183078529 (564h13m44.535863658s) [noise=true , address=45.91.116.8:4045 , piece_id=CYZ5LMTPWHDFGWPNAHTD4NCBGIIIY76LS2D2EXH7TQ4NHFWEOH4Q, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.183177015 (564h13m44.535962142s) [noise=true , address=145.79.227.4:4187 , piece_id=RAN3UTPCEQ3523F5ZCEVF64CWKK7RJWMUD5OVVLF6IBGF7TJK7AA, seg_pos={0 0}, upload=496551]: Uploading piece... Sep 22 02:22:09.288931400 (564h13m44.641716548s) [upload=496551]: Next returned nil segment Sep 22 02:22:09.288953130 (564h13m44.641738309s) [upload=496551]: Waiting for error group managing segments... Sep 22 02:22:09.297528322 (564h13m44.65031345s) [noise=true , address=45.91.116.9:4261 , piece_id=37PHHPAWDXLBPE56XVH46QPXTXNSX7GRXMSPVANWJUBV5CSPYEPQ, seg_pos={0 0}, upload=496551]: Done uploading piece. err: <nil> Sep 22 02:22:09.298240008 (564h13m44.651025146s) [noise=true , address=45.91.116.6:4975 , piece_id=PBXYHGT3I4P7ZJ3HASGR5IHI5NVFGA7R5VYJBZ7FBXEB7BKXKCKA, seg_pos={0 0}, upload=496551]: Done uploading piece. err: <nil> Sep 22 02:22:09.298321902 (564h13m44.651107029s) [noise=true , address=45.91.116.14:4191 , piece_id=URYYG4NYG6JI754Y37PT27MNYF3YCQHNIFKQEW3I7WDCMNL3QUIA, seg_pos={0 0}, upload=496551]: Done uploading piece. err: <nil> Sep 22 02:22:09.298893364 (564h13m44.651678492s) [noise=true , address=77.90.31.12:4590 , piece_id=3GANTEOT4LO65G4PDSS6DVVCPYOERTZZGGDPZDA6EB376UYMRVVQ, seg_pos={0 0}, upload=496551]: Done uploading piece. err: <nil> Sep 22 02:22:09.299154804 (564h13m44.651939933s) [noise=true , address=45.91.116.4:4938 , piece_id=R3NOVEHFPRHZ73CDORFOCCHTOOYFIJWFVJJYYQ4VKDOB6OQPFHRA, seg_pos={0 0}, upload=496551]: Done uploading piece. err: <nil> Sep 22 02:22:09.299479363 (564h13m44.652264501s) [noise=true , address=45.91.116.11:4117 , piece_id=RFKTUYY272OABR56VGDXBG7YKTAAIBTMP5IUBV6YMMVOIHIBPZCA, seg_pos={0 0}, upload=496551]: Done uploading piece. err: <nil> Sep 22 02:22:09.299893821 (564h13m44.65267895s) [noise=true , address=45.91.116.8:4045 , piece_id=CYZ5LMTPWHDFGWPNAHTD4NCBGIIIY76LS2D2EXH7TQ4NHFWEOH4Q, seg_pos={0 0}, upload=496551]: Done uploading piece. err: <nil> Sep 22 02:22:09.301609902 (564h13m44.65439503s) [noise=true , address=77.90.31.21:4878 , piece_id=4RFPKSZEZPH3Y7QIQFQMF24EABH52FEUYCNBDIWHG5G4NC2ZZBQQ, seg_pos={0 0}, upload=496551]: Done uploading piece. err: <nil> Sep 22 02:22:09.308864345 (564h13m44.661649483s) [noise=true , address=45.91.116.5:4298 , piece_id=OGPIA3MYBZS2RKRI7G6YQUNUUDIKJ52DIERC5KRZRZFGTOUILKQA, seg_pos={0 0}, upload=496551]: Done uploading piece. err: <nil> Sep 22 02:22:39.055394557 (564h14m14.408179685s) [noise=true , address=145.79.227.13:4802 , piece_id=LTHSKOSHKC2YYCJE5XO5XII2WQZA5UEONG5WVXXEB5J33PUGOHOA, seg_pos={0 0}, upload=496551]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:22:39.055501067 (564h14m14.408286195s) [noise=true , address=145.79.227.2:4113 , piece_id=LPHCGCOYFXQEBKMPNG3BXK67FJNCRI27C6YQCRJ5VJCOOIXLLZ6Q, seg_pos={0 0}, upload=496551]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: rpc: dial tcp 145.79.227.2:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:22:39.055676616 (564h14m14.408461744s) [noise=true , address=145.79.227.5:4223 , piece_id=BQJUF4L47G5MNFL3OMGH2AZXLFTBG52VDLIDWZXSMTGW4D5OAENA, seg_pos={0 0}, upload=496551]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: rpc: dial tcp 145.79.227.5:4223: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:22:39.055725277 (564h14m14.408510405s) [noise=true , address=145.79.227.11:4731 , piece_id=VZWBU4ZXL6M5KPQDET7RGZR5BS5GB2UJXMJFXQUKT6Z5OCKE3KJQ, seg_pos={0 0}, upload=496551]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:22:39.055746828 (564h14m14.408531956s) [noise=true , address=145.79.227.19:4077 , piece_id=FW6272PRMJWOZWVWPTNEK3HCGIODO7YQ2OPFYZTDZ52UAR6MKUWA, seg_pos={0 0}, upload=496551]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:145.79.227.19:4077): protocol: rpc: dial tcp 145.79.227.19:4077: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:22:39.055805578 (564h14m14.408590706s) [noise=true , address=145.79.227.4:4187 , piece_id=RAN3UTPCEQ3523F5ZCEVF64CWKK7RJWMUD5OVVLF6IBGF7TJK7AA, seg_pos={0 0}, upload=496551]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:22:39.055819244 (564h14m14.408604381s) [noise=true , address=145.79.227.6:4258 , piece_id=BQPKF37ODCA56JJ2RHLPNBSWMADXOWZRR6TGHVCY4ZB4TPCZC77A, seg_pos={0 0}, upload=496551]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: rpc: dial tcp 145.79.227.6:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:22:39.055864699 (564h14m14.408649826s) [seg_pos={0 0}, upload=496551]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 22 02:22:39.070591375 (564h14m14.423376502s) [upload=496551]: Done uploading segments... Sep 22 02:22:39.070607855 (564h14m14.423392984s) [upload=496551]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.016866647s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................X.X.........X......] duration: 392.753727ms FAIL (2025-09-22 02:22:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 392.769658ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.......................................XXX....XXX.XXXXX..X.X] duration: 30.01914849s FAIL (2025-09-22 02:22:03 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:21:33.944589669 (564h13m9.297374798s) [upload=496540]: Starting upload Sep 22 02:21:33.944663898 (564h13m9.297449027s) [upload=496540]: Done starting upload Sep 22 02:21:33.944906564 (564h13m9.297691692s) [upload=496540]: Uploading segments... Sep 22 02:21:33.945217749 (564h13m9.298002886s) [upload=496540]: Got next segment. Inline: false Sep 22 02:21:33.945299432 (564h13m9.298084559s) [upload=496540]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:21:33.945309240 (564h13m9.298094368s) [upload=496540]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:21:34.041195126 (564h13m9.393980255s) [seg_pos={0 0}, upload=496540]: Begin upload segment... Sep 22 02:21:34.041238678 (564h13m9.394023796s) [seg_pos={0 0}, upload=496540]: Done begin upload segment. Sep 22 02:21:34.041266080 (564h13m9.394051198s) [noise=true , address=145.79.227.13:4803 , piece_id=TL3377L56FO2PWLOTBMPGNL3445WYJRFY6RIWVIBYTUCUSRZRO6A, seg_pos={0 0}, upload=496540]: Uploading piece... Sep 22 02:21:34.041278423 (564h13m9.394063551s) [noise=true , address=77.90.31.13:4626 , piece_id=IXDFA6LTC4FP2VZMK6R652WMSTWGT2LCXTDID4XWYBG27RVRI77A, seg_pos={0 0}, upload=496540]: Uploading piece... Sep 22 02:21:34.041739868 (564h13m9.394524996s) [noise=true , address=45.91.116.13:4154 , piece_id=DFIKUNG6RQBOX4FBFV3BLFETVZETIJ7236GYJO52CBD56WDM7DCA, seg_pos={0 0}, upload=496540]: Uploading piece... Sep 22 02:21:34.041757000 (564h13m9.394542128s) [noise=true , address=77.90.31.19:4843 , piece_id=V62XE5JEZPJPTGKXKBIK7IJCWIR4LF67OAM7ZFZONDF3JRP3AO6A, seg_pos={0 0}, upload=496540]: Uploading piece... Sep 22 02:21:34.042095005 (564h13m9.394880123s) [noise=true , address=145.79.227.17:4972 , piece_id=U2OYUKDFBUUGFXP4HVITLYDDBDUVBTFWKMHDMF3OZ3DB2K26VUJQ, seg_pos={0 0}, upload=496540]: Uploading piece... Sep 22 02:21:34.042106316 (564h13m9.394891445s) [noise=true , address=77.90.31.16:4734 , piece_id=36MKZEFEXYGUJ4TDTWUNYDRJZMNHNBD326SUJGNNIZAYW74M3TKQ, seg_pos={0 0}, upload=496540]: Uploading piece... Sep 22 02:21:34.042421868 (564h13m9.395206997s) [noise=true , address=145.79.227.3:4150 , piece_id=LOE63IMPS6KW2HYBXX5OVIEG557TLHB45Y4QGK4GIYUPPEBMMB5Q, seg_pos={0 0}, upload=496540]: Uploading piece... Sep 22 02:21:34.042516265 (564h13m9.395301383s) [noise=true , address=45.91.116.13:4155 , piece_id=PL2VQENYNOYD74WJMGM6JABB4UXWBG3UDTUHJUZFE46PSLUXSWIQ, seg_pos={0 0}, upload=496540]: Uploading piece... Sep 22 02:21:34.042763669 (564h13m9.395548788s) [noise=true , address=45.91.116.5:4297 , piece_id=XJC24IWRTAGOKCRBNKRYP7U2IYK7I6A4QVLPP7AU27B4IDP4A2CA, seg_pos={0 0}, upload=496540]: Uploading piece... Sep 22 02:21:34.042860370 (564h13m9.395645489s) [noise=true , address=77.90.31.21:4879 , piece_id=G3RJOZ5TJCKAOOPLUOYTU7PE5IILN44JOF2ZKCCHVBSKS5ADI5WA, seg_pos={0 0}, upload=496540]: Uploading piece... Sep 22 02:21:34.043183969 (564h13m9.395969097s) [noise=true , address=145.79.227.18:4007 , piece_id=WLQJNVSGBJGMEVGZML6Y56T4RCQIZIBTHWQ4NGQ6KIQH6V2XSUUQ, seg_pos={0 0}, upload=496540]: Uploading piece... Sep 22 02:21:34.043530809 (564h13m9.396315937s) [noise=true , address=45.91.116.7:4009 , piece_id=YJBTHVLQ5CWBNZJYRSSO26YJT74564RLZPPJEDTRL4ZKZAIMK57A, seg_pos={0 0}, upload=496540]: Uploading piece... Sep 22 02:21:34.043876037 (564h13m9.396661156s) [noise=true , address=45.91.116.14:4189 , piece_id=GIFAKEOJ3JCPKGGXJDHIGUXFQRMG53K67KYRKVV4NFIBZE44FOBA, seg_pos={0 0}, upload=496540]: Uploading piece... Sep 22 02:21:34.044247705 (564h13m9.397032833s) [noise=true , address=45.91.116.9:4261 , piece_id=QJI57627D2VWI7ORNGDWOJERYEDYJW4WXK3YVYNXZID37RSMRHOQ, seg_pos={0 0}, upload=496540]: Uploading piece... Sep 22 02:21:34.046789365 (564h13m9.399574492s) [noise=true , address=77.90.31.18:4805 , piece_id=22TN2DGKEHWOCMLO37UBOAH33BN4SPUN5C5SKKO7HPC2JTEJWGNQ, seg_pos={0 0}, upload=496540]: Uploading piece... Sep 22 02:21:34.047369733 (564h13m9.400154851s) [noise=true , address=145.79.227.16:4910 , piece_id=BXVGI66YF5HDTOOSBNW7CXOKLC3USR5TZCPT7EWY7OFE7NMOXGUA, seg_pos={0 0}, upload=496540]: Uploading piece... Sep 22 02:21:34.117432221 (564h13m9.470217319s) [upload=496540]: Next returned nil segment Sep 22 02:21:34.117467296 (564h13m9.470252404s) [upload=496540]: Waiting for error group managing segments... Sep 22 02:21:34.125399321 (564h13m9.478184429s) [noise=true , address=45.91.116.14:4189 , piece_id=GIFAKEOJ3JCPKGGXJDHIGUXFQRMG53K67KYRKVV4NFIBZE44FOBA, seg_pos={0 0}, upload=496540]: Done uploading piece. err: <nil> Sep 22 02:21:34.125593876 (564h13m9.478378974s) [noise=true , address=45.91.116.9:4261 , piece_id=QJI57627D2VWI7ORNGDWOJERYEDYJW4WXK3YVYNXZID37RSMRHOQ, seg_pos={0 0}, upload=496540]: Done uploading piece. err: <nil> Sep 22 02:21:34.126391713 (564h13m9.479176821s) [noise=true , address=45.91.116.13:4155 , piece_id=PL2VQENYNOYD74WJMGM6JABB4UXWBG3UDTUHJUZFE46PSLUXSWIQ, seg_pos={0 0}, upload=496540]: Done uploading piece. err: <nil> Sep 22 02:21:34.127818271 (564h13m9.480603379s) [noise=true , address=45.91.116.7:4009 , piece_id=YJBTHVLQ5CWBNZJYRSSO26YJT74564RLZPPJEDTRL4ZKZAIMK57A, seg_pos={0 0}, upload=496540]: Done uploading piece. err: <nil> Sep 22 02:21:34.128011603 (564h13m9.480796711s) [noise=true , address=45.91.116.5:4297 , piece_id=XJC24IWRTAGOKCRBNKRYP7U2IYK7I6A4QVLPP7AU27B4IDP4A2CA, seg_pos={0 0}, upload=496540]: Done uploading piece. err: <nil> Sep 22 02:21:34.128246705 (564h13m9.481031803s) [noise=true , address=77.90.31.21:4879 , piece_id=G3RJOZ5TJCKAOOPLUOYTU7PE5IILN44JOF2ZKCCHVBSKS5ADI5WA, seg_pos={0 0}, upload=496540]: Done uploading piece. err: <nil> Sep 22 02:21:34.128436510 (564h13m9.481221609s) [noise=true , address=77.90.31.18:4805 , piece_id=22TN2DGKEHWOCMLO37UBOAH33BN4SPUN5C5SKKO7HPC2JTEJWGNQ, seg_pos={0 0}, upload=496540]: Done uploading piece. err: <nil> Sep 22 02:21:34.129495097 (564h13m9.482280195s) [noise=true , address=77.90.31.13:4626 , piece_id=IXDFA6LTC4FP2VZMK6R652WMSTWGT2LCXTDID4XWYBG27RVRI77A, seg_pos={0 0}, upload=496540]: Done uploading piece. err: <nil> Sep 22 02:21:34.131205768 (564h13m9.483990876s) [noise=true , address=77.90.31.19:4843 , piece_id=V62XE5JEZPJPTGKXKBIK7IJCWIR4LF67OAM7ZFZONDF3JRP3AO6A, seg_pos={0 0}, upload=496540]: Done uploading piece. err: <nil> Sep 22 02:21:34.136970405 (564h13m9.489755503s) [noise=true , address=77.90.31.16:4734 , piece_id=36MKZEFEXYGUJ4TDTWUNYDRJZMNHNBD326SUJGNNIZAYW74M3TKQ, seg_pos={0 0}, upload=496540]: Done uploading piece. err: <nil> Sep 22 02:21:34.139966168 (564h13m9.492751276s) [noise=true , address=45.91.116.13:4154 , piece_id=DFIKUNG6RQBOX4FBFV3BLFETVZETIJ7236GYJO52CBD56WDM7DCA, seg_pos={0 0}, upload=496540]: Done uploading piece. err: <nil> Sep 22 02:22:03.931324111 (564h13m39.28410921s) [noise=true , address=145.79.227.3:4150 , piece_id=LOE63IMPS6KW2HYBXX5OVIEG557TLHB45Y4QGK4GIYUPPEBMMB5Q, seg_pos={0 0}, upload=496540]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: rpc: dial tcp 145.79.227.3:4150: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:22:03.931420974 (564h13m39.284206082s) [noise=true , address=145.79.227.16:4910 , piece_id=BXVGI66YF5HDTOOSBNW7CXOKLC3USR5TZCPT7EWY7OFE7NMOXGUA, seg_pos={0 0}, upload=496540]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:22:03.931507657 (564h13m39.284292755s) [noise=true , address=145.79.227.17:4972 , piece_id=U2OYUKDFBUUGFXP4HVITLYDDBDUVBTFWKMHDMF3OZ3DB2K26VUJQ, seg_pos={0 0}, upload=496540]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: rpc: dial tcp 145.79.227.17:4972: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:22:03.931554625 (564h13m39.284339723s) [noise=true , address=145.79.227.18:4007 , piece_id=WLQJNVSGBJGMEVGZML6Y56T4RCQIZIBTHWQ4NGQ6KIQH6V2XSUUQ, seg_pos={0 0}, upload=496540]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: rpc: dial tcp 145.79.227.18:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:22:03.931566978 (564h13m39.284352076s) [noise=true , address=145.79.227.13:4803 , piece_id=TL3377L56FO2PWLOTBMPGNL3445WYJRFY6RIWVIBYTUCUSRZRO6A, seg_pos={0 0}, upload=496540]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: rpc: dial tcp 145.79.227.13:4803: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:22:03.931610539 (564h13m39.284395647s) [seg_pos={0 0}, upload=496540]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:22:03.948698947 (564h13m39.301484055s) [upload=496540]: Done uploading segments... Sep 22 02:22:03.948717892 (564h13m39.301503s) [upload=496540]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.019162265s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [..........................................X.XXXXXX..X.......] duration: 450.037479ms FAIL (2025-09-22 02:21:40 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== m30.954857157s) [seg_pos={0 0}, upload=496461]: Begin upload segment... Sep 22 02:15:55.602116061 (564h7m30.954901189s) [seg_pos={0 0}, upload=496461]: Done begin upload segment. Sep 22 02:15:55.602137471 (564h7m30.954922599s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602154613 (564h7m30.954939742s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602716609 (564h7m30.955501736s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602736295 (564h7m30.955521423s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603134873 (564h7m30.955920001s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603640462 (564h7m30.95642559s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604036745 (564h7m30.956821874s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604397323 (564h7m30.95718245s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604828131 (564h7m30.957613268s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605250674 (564h7m30.958035802s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605663548 (564h7m30.958448676s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606022221 (564h7m30.958807349s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606387297 (564h7m30.959172425s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606869060 (564h7m30.959654198s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607319626 (564h7m30.960104755s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607717662 (564h7m30.960502801s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608211139 (564h7m30.960996268s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608652857 (564h7m30.961437986s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.704831022 (564h7m31.05761613s) [upload=496461]: Next returned nil segment Sep 22 02:15:55.704846542 (564h7m31.05763165s) [upload=496461]: Waiting for error group managing segments... Sep 22 02:15:55.712709156 (564h7m31.065494285s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.713081926 (564h7m31.065867054s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.716351481 (564h7m31.06913661s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717069920 (564h7m31.069855048s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717303427 (564h7m31.070088556s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725190529 (564h7m31.077975657s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725273014 (564h7m31.078058142s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732220611 (564h7m31.085005749s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732337590 (564h7m31.085122719s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.734847580 (564h7m31.087632718s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.738226892 (564h7m31.09101203s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:16:55.564886089 (564h8m30.917671218s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1AowWwVvxzmnW5qwbqbMBVbvNoigffjAN51LQp8e6XA9CZkeXL, address:145.79.227.12:4790): protocol: rpc: dial tcp 145.79.227.12:4790: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564944549 (564h8m30.917729677s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564994583 (564h8m30.917779711s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565058042 (564h8m30.91784317s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1zywsXJnw5gfpUu7mC8p7UE7ev5HexPu5nSpUH1x2pzar99T97, address:145.79.227.8:4349): protocol: rpc: dial tcp 145.79.227.8:4349: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565089361 (564h8m30.917874489s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:145.79.227.14:4847): protocol: rpc: dial tcp 145.79.227.14:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565111382 (564h8m30.91789651s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12Pxos2xcjHwcYyQnXeRL3PL72G9pXaAnBs8XnkVFkxegDRPrqp, address:145.79.227.2:4127): protocol: rpc: dial tcp 145.79.227.2:4127: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565129266 (564h8m30.917914394s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12gs9LRE3oUp1jupCw5uDvRtkNcahbq2d4VTYQsdKr6atk6vGJx, address:145.79.227.19:4095): protocol: rpc: dial tcp 145.79.227.19:4095: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565148231 (564h8m30.917933359s) [seg_pos={0 0}, upload=496461]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:16:55.580701268 (564h8m30.933486366s) [upload=496461]: Done uploading segments... Sep 22 02:16:55.580714373 (564h8m30.93349947s) [upload=496461]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 450.052877ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................................X.X.........X.....] duration: 361.106499ms FAIL (2025-09-22 02:21:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 361.122408ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [........................................XXX....XXX.XXXXX..X.] duration: 399.059102ms FAIL (2025-09-22 02:21:05 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:20:00.590299489 (564h11m35.943084597s) [upload=496519]: Starting upload Sep 22 02:20:00.590355544 (564h11m35.943140642s) [upload=496519]: Done starting upload Sep 22 02:20:00.590409014 (564h11m35.943194112s) [upload=496519]: Uploading segments... Sep 22 02:20:00.590579454 (564h11m35.943364562s) [upload=496519]: Got next segment. Inline: false Sep 22 02:20:00.590614099 (564h11m35.943399207s) [upload=496519]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:20:00.590618026 (564h11m35.943403125s) [upload=496519]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:20:00.615037983 (564h11m35.967823091s) [seg_pos={0 0}, upload=496519]: Begin upload segment... Sep 22 02:20:00.615073289 (564h11m35.967858397s) [seg_pos={0 0}, upload=496519]: Done begin upload segment. Sep 22 02:20:00.615097675 (564h11m35.967882784s) [noise=true , address=145.79.227.11:4729 , piece_id=HRIXDZ7WATCOJCMUJ74JVWTE3YKDCJRYQHMQUSRGXJRX7GJRTCYQ, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.615114988 (564h11m35.967900085s) [noise=true , address=45.91.116.8:4048 , piece_id=ACTWP4FJPDW2GXEWBRW6CXCNK7W2YQ5NMEBQR6SURCGDYB6OHOIQ, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.615403308 (564h11m35.968188416s) [noise=true , address=45.91.116.8:4047 , piece_id=VA5OXAVATNGIPOZ4Z7V2SBCANJSG4YU5N4DWUZ27I5TOVPMWYB3Q, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.615494550 (564h11m35.968279658s) [noise=true , address=45.91.116.14:4190 , piece_id=ALAIMRP6VXRIJK5MXQINBNR4PIJAWR3K5DWGC6VKYYYANMFJ7UMQ, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.615671872 (564h11m35.96845698s) [noise=true , address=45.91.116.11:4117 , piece_id=7W5B2UPMJDZZKHSOYTGGDXJ32X7AF25C6OOXAL4Y2YYP2R2JGUZQ, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.615779063 (564h11m35.968564161s) [noise=true , address=77.90.31.15:4699 , piece_id=2UD5RG27NTFJF2ATOHXIGB53QKMN2DUJYHCTQFDHIBJVER5WZHYQ, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.615901413 (564h11m35.968686521s) [noise=true , address=145.79.227.5:4222 , piece_id=7MIKE4DN7D554ROEHQFOZFRWATHGQWRVTCWIPZ7X42BVC6YMOZ2A, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.616068788 (564h11m35.968853886s) [noise=true , address=45.91.116.13:4154 , piece_id=UN7J7A65OPTBILWIDOOTMWTAWXO2BKSINOL7FQ53WSTVBJEE45KQ, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.616152264 (564h11m35.968937372s) [noise=true , address=45.91.116.9:4262 , piece_id=Y2YMKMX3KTRUX5DL4H3LG5BXQ4FGPRBISWCIIMEEB273OXUOCK2Q, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.616351518 (564h11m35.969136626s) [noise=true , address=145.79.227.18:4006 , piece_id=XSOO52PBFUPTUK4OQD3K2WZSL5F57LPYREVFYF72EX2RCZVL62BQ, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.616383117 (564h11m35.969168215s) [noise=true , address=77.90.31.19:4842 , piece_id=YZN7TNH7UA4RHXEQ3PWL6ZUHHKCIXHODLWWKUKJNHJDN7A3FK4WA, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.616614281 (564h11m35.969399389s) [noise=true , address=45.91.116.12:4227 , piece_id=RBX5EOAUEONFVGFUVUYYFCKORPEAW4ZSVXAXT5X465BC2VOGMQ7Q, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.617835663 (564h11m35.970620762s) [noise=true , address=45.91.116.12:4226 , piece_id=5W3UXRF7Y7PDAYYZ7UDVKM4VU6UHBIJWFDFOMNR5RNNTZPDYUEQQ, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.618142709 (564h11m35.970927807s) [noise=true , address=45.91.116.13:4153 , piece_id=ZFD3QMFUQIIX74ZJYYHUUQFO6433AH2ESO6Y6W65SRHSDFZR5QAQ, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.618504478 (564h11m35.971289586s) [noise=true , address=145.79.227.13:4801 , piece_id=5G6C45PY72GZZOBN7CODUFQ474VTNC5U6QIQVTPJNZKHZYOGKVHQ, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.618815281 (564h11m35.971600389s) [noise=true , address=45.91.116.7:4009 , piece_id=OA6AV7X76Z3G3ZV2EYLZSYAWZZTPD72H4FX2NK7NLYH6DSOKDOUA, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.695412773 (564h11m36.048197881s) [upload=496519]: Next returned nil segment Sep 22 02:20:00.695455443 (564h11m36.048240551s) [upload=496519]: Waiting for error group managing segments... Sep 22 02:20:00.704339253 (564h11m36.057124361s) [noise=true , address=45.91.116.8:4047 , piece_id=VA5OXAVATNGIPOZ4Z7V2SBCANJSG4YU5N4DWUZ27I5TOVPMWYB3Q, seg_pos={0 0}, upload=496519]: Done uploading piece. err: <nil> Sep 22 02:20:00.704633887 (564h11m36.057418985s) [noise=true , address=45.91.116.14:4190 , piece_id=ALAIMRP6VXRIJK5MXQINBNR4PIJAWR3K5DWGC6VKYYYANMFJ7UMQ, seg_pos={0 0}, upload=496519]: Done uploading piece. err: <nil> Sep 22 02:20:00.704835375 (564h11m36.057620483s) [noise=true , address=45.91.116.13:4154 , piece_id=UN7J7A65OPTBILWIDOOTMWTAWXO2BKSINOL7FQ53WSTVBJEE45KQ, seg_pos={0 0}, upload=496519]: Done uploading piece. err: <nil> Sep 22 02:20:00.704994984 (564h11m36.057780092s) [noise=true , address=45.91.116.11:4117 , piece_id=7W5B2UPMJDZZKHSOYTGGDXJ32X7AF25C6OOXAL4Y2YYP2R2JGUZQ, seg_pos={0 0}, upload=496519]: Done uploading piece. err: <nil> Sep 22 02:20:00.705242779 (564h11m36.058027887s) [noise=true , address=45.91.116.12:4226 , piece_id=5W3UXRF7Y7PDAYYZ7UDVKM4VU6UHBIJWFDFOMNR5RNNTZPDYUEQQ, seg_pos={0 0}, upload=496519]: Done uploading piece. err: <nil> Sep 22 02:20:00.705328820 (564h11m36.058113928s) [noise=true , address=45.91.116.13:4153 , piece_id=ZFD3QMFUQIIX74ZJYYHUUQFO6433AH2ESO6Y6W65SRHSDFZR5QAQ, seg_pos={0 0}, upload=496519]: Done uploading piece. err: <nil> Sep 22 02:20:00.705605951 (564h11m36.058391049s) [noise=true , address=45.91.116.12:4227 , piece_id=RBX5EOAUEONFVGFUVUYYFCKORPEAW4ZSVXAXT5X465BC2VOGMQ7Q, seg_pos={0 0}, upload=496519]: Done uploading piece. err: <nil> Sep 22 02:20:00.705704075 (564h11m36.058489183s) [noise=true , address=77.90.31.15:4699 , piece_id=2UD5RG27NTFJF2ATOHXIGB53QKMN2DUJYHCTQFDHIBJVER5WZHYQ, seg_pos={0 0}, upload=496519]: Done uploading piece. err: <nil> Sep 22 02:20:00.705994099 (564h11m36.058779207s) [noise=true , address=45.91.116.9:4262 , piece_id=Y2YMKMX3KTRUX5DL4H3LG5BXQ4FGPRBISWCIIMEEB273OXUOCK2Q, seg_pos={0 0}, upload=496519]: Done uploading piece. err: <nil> Sep 22 02:20:00.706022643 (564h11m36.058807751s) [noise=true , address=77.90.31.19:4842 , piece_id=YZN7TNH7UA4RHXEQ3PWL6ZUHHKCIXHODLWWKUKJNHJDN7A3FK4WA, seg_pos={0 0}, upload=496519]: Done uploading piece. err: <nil> Sep 22 02:20:00.706358454 (564h11m36.059143551s) [noise=true , address=45.91.116.7:4009 , piece_id=OA6AV7X76Z3G3ZV2EYLZSYAWZZTPD72H4FX2NK7NLYH6DSOKDOUA, seg_pos={0 0}, upload=496519]: Done uploading piece. err: <nil> Sep 22 02:20:00.706545063 (564h11m36.059330162s) [noise=true , address=45.91.116.8:4048 , piece_id=ACTWP4FJPDW2GXEWBRW6CXCNK7W2YQ5NMEBQR6SURCGDYB6OHOIQ, seg_pos={0 0}, upload=496519]: Done uploading piece. err: <nil> Sep 22 02:20:30.577767801 (564h12m5.9305529s) [noise=true , address=145.79.227.13:4801 , piece_id=5G6C45PY72GZZOBN7CODUFQ474VTNC5U6QIQVTPJNZKHZYOGKVHQ, seg_pos={0 0}, upload=496519]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: rpc: dial tcp 145.79.227.13:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:20:30.577816704 (564h12m5.930601841s) [noise=true , address=145.79.227.18:4006 , piece_id=XSOO52PBFUPTUK4OQD3K2WZSL5F57LPYREVFYF72EX2RCZVL62BQ, seg_pos={0 0}, upload=496519]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:20:30.577872568 (564h12m5.930657676s) [noise=true , address=145.79.227.11:4729 , piece_id=HRIXDZ7WATCOJCMUJ74JVWTE3YKDCJRYQHMQUSRGXJRX7GJRTCYQ, seg_pos={0 0}, upload=496519]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:20:30.577907233 (564h12m5.930692371s) [noise=true , address=145.79.227.5:4222 , piece_id=7MIKE4DN7D554ROEHQFOZFRWATHGQWRVTCWIPZ7X42BVC6YMOZ2A, seg_pos={0 0}, upload=496519]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:20:30.577990379 (564h12m5.930775507s) [seg_pos={0 0}, upload=496519]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:20:30.594027343 (564h12m5.946812472s) [upload=496519]: Done uploading segments... Sep 22 02:20:30.594065766 (564h12m5.946850893s) [upload=496519]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 399.077255ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [...........................................X.XXXXXX..X......] duration: 493.97611ms FAIL (2025-09-22 02:20:35 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== m30.954857157s) [seg_pos={0 0}, upload=496461]: Begin upload segment... Sep 22 02:15:55.602116061 (564h7m30.954901189s) [seg_pos={0 0}, upload=496461]: Done begin upload segment. Sep 22 02:15:55.602137471 (564h7m30.954922599s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602154613 (564h7m30.954939742s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602716609 (564h7m30.955501736s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602736295 (564h7m30.955521423s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603134873 (564h7m30.955920001s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603640462 (564h7m30.95642559s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604036745 (564h7m30.956821874s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604397323 (564h7m30.95718245s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604828131 (564h7m30.957613268s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605250674 (564h7m30.958035802s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605663548 (564h7m30.958448676s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606022221 (564h7m30.958807349s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606387297 (564h7m30.959172425s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606869060 (564h7m30.959654198s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607319626 (564h7m30.960104755s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607717662 (564h7m30.960502801s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608211139 (564h7m30.960996268s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608652857 (564h7m30.961437986s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.704831022 (564h7m31.05761613s) [upload=496461]: Next returned nil segment Sep 22 02:15:55.704846542 (564h7m31.05763165s) [upload=496461]: Waiting for error group managing segments... Sep 22 02:15:55.712709156 (564h7m31.065494285s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.713081926 (564h7m31.065867054s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.716351481 (564h7m31.06913661s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717069920 (564h7m31.069855048s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717303427 (564h7m31.070088556s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725190529 (564h7m31.077975657s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725273014 (564h7m31.078058142s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732220611 (564h7m31.085005749s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732337590 (564h7m31.085122719s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.734847580 (564h7m31.087632718s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.738226892 (564h7m31.09101203s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:16:55.564886089 (564h8m30.917671218s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1AowWwVvxzmnW5qwbqbMBVbvNoigffjAN51LQp8e6XA9CZkeXL, address:145.79.227.12:4790): protocol: rpc: dial tcp 145.79.227.12:4790: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564944549 (564h8m30.917729677s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564994583 (564h8m30.917779711s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565058042 (564h8m30.91784317s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1zywsXJnw5gfpUu7mC8p7UE7ev5HexPu5nSpUH1x2pzar99T97, address:145.79.227.8:4349): protocol: rpc: dial tcp 145.79.227.8:4349: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565089361 (564h8m30.917874489s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:145.79.227.14:4847): protocol: rpc: dial tcp 145.79.227.14:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565111382 (564h8m30.91789651s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12Pxos2xcjHwcYyQnXeRL3PL72G9pXaAnBs8XnkVFkxegDRPrqp, address:145.79.227.2:4127): protocol: rpc: dial tcp 145.79.227.2:4127: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565129266 (564h8m30.917914394s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12gs9LRE3oUp1jupCw5uDvRtkNcahbq2d4VTYQsdKr6atk6vGJx, address:145.79.227.19:4095): protocol: rpc: dial tcp 145.79.227.19:4095: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565148231 (564h8m30.917933359s) [seg_pos={0 0}, upload=496461]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:16:55.580701268 (564h8m30.933486366s) [upload=496461]: Done uploading segments... Sep 22 02:16:55.580714373 (564h8m30.93349947s) [upload=496461]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 493.999012ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................................X.X.........X....] duration: 445.999729ms FAIL (2025-09-22 02:20:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 446.010859ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.........................................XXX....XXX.XXXXX..X] duration: 30.019043076s FAIL (2025-09-22 02:20:30 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:20:00.590299489 (564h11m35.943084597s) [upload=496519]: Starting upload Sep 22 02:20:00.590355544 (564h11m35.943140642s) [upload=496519]: Done starting upload Sep 22 02:20:00.590409014 (564h11m35.943194112s) [upload=496519]: Uploading segments... Sep 22 02:20:00.590579454 (564h11m35.943364562s) [upload=496519]: Got next segment. Inline: false Sep 22 02:20:00.590614099 (564h11m35.943399207s) [upload=496519]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:20:00.590618026 (564h11m35.943403125s) [upload=496519]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:20:00.615037983 (564h11m35.967823091s) [seg_pos={0 0}, upload=496519]: Begin upload segment... Sep 22 02:20:00.615073289 (564h11m35.967858397s) [seg_pos={0 0}, upload=496519]: Done begin upload segment. Sep 22 02:20:00.615097675 (564h11m35.967882784s) [noise=true , address=145.79.227.11:4729 , piece_id=HRIXDZ7WATCOJCMUJ74JVWTE3YKDCJRYQHMQUSRGXJRX7GJRTCYQ, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.615114988 (564h11m35.967900085s) [noise=true , address=45.91.116.8:4048 , piece_id=ACTWP4FJPDW2GXEWBRW6CXCNK7W2YQ5NMEBQR6SURCGDYB6OHOIQ, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.615403308 (564h11m35.968188416s) [noise=true , address=45.91.116.8:4047 , piece_id=VA5OXAVATNGIPOZ4Z7V2SBCANJSG4YU5N4DWUZ27I5TOVPMWYB3Q, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.615494550 (564h11m35.968279658s) [noise=true , address=45.91.116.14:4190 , piece_id=ALAIMRP6VXRIJK5MXQINBNR4PIJAWR3K5DWGC6VKYYYANMFJ7UMQ, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.615671872 (564h11m35.96845698s) [noise=true , address=45.91.116.11:4117 , piece_id=7W5B2UPMJDZZKHSOYTGGDXJ32X7AF25C6OOXAL4Y2YYP2R2JGUZQ, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.615779063 (564h11m35.968564161s) [noise=true , address=77.90.31.15:4699 , piece_id=2UD5RG27NTFJF2ATOHXIGB53QKMN2DUJYHCTQFDHIBJVER5WZHYQ, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.615901413 (564h11m35.968686521s) [noise=true , address=145.79.227.5:4222 , piece_id=7MIKE4DN7D554ROEHQFOZFRWATHGQWRVTCWIPZ7X42BVC6YMOZ2A, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.616068788 (564h11m35.968853886s) [noise=true , address=45.91.116.13:4154 , piece_id=UN7J7A65OPTBILWIDOOTMWTAWXO2BKSINOL7FQ53WSTVBJEE45KQ, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.616152264 (564h11m35.968937372s) [noise=true , address=45.91.116.9:4262 , piece_id=Y2YMKMX3KTRUX5DL4H3LG5BXQ4FGPRBISWCIIMEEB273OXUOCK2Q, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.616351518 (564h11m35.969136626s) [noise=true , address=145.79.227.18:4006 , piece_id=XSOO52PBFUPTUK4OQD3K2WZSL5F57LPYREVFYF72EX2RCZVL62BQ, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.616383117 (564h11m35.969168215s) [noise=true , address=77.90.31.19:4842 , piece_id=YZN7TNH7UA4RHXEQ3PWL6ZUHHKCIXHODLWWKUKJNHJDN7A3FK4WA, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.616614281 (564h11m35.969399389s) [noise=true , address=45.91.116.12:4227 , piece_id=RBX5EOAUEONFVGFUVUYYFCKORPEAW4ZSVXAXT5X465BC2VOGMQ7Q, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.617835663 (564h11m35.970620762s) [noise=true , address=45.91.116.12:4226 , piece_id=5W3UXRF7Y7PDAYYZ7UDVKM4VU6UHBIJWFDFOMNR5RNNTZPDYUEQQ, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.618142709 (564h11m35.970927807s) [noise=true , address=45.91.116.13:4153 , piece_id=ZFD3QMFUQIIX74ZJYYHUUQFO6433AH2ESO6Y6W65SRHSDFZR5QAQ, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.618504478 (564h11m35.971289586s) [noise=true , address=145.79.227.13:4801 , piece_id=5G6C45PY72GZZOBN7CODUFQ474VTNC5U6QIQVTPJNZKHZYOGKVHQ, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.618815281 (564h11m35.971600389s) [noise=true , address=45.91.116.7:4009 , piece_id=OA6AV7X76Z3G3ZV2EYLZSYAWZZTPD72H4FX2NK7NLYH6DSOKDOUA, seg_pos={0 0}, upload=496519]: Uploading piece... Sep 22 02:20:00.695412773 (564h11m36.048197881s) [upload=496519]: Next returned nil segment Sep 22 02:20:00.695455443 (564h11m36.048240551s) [upload=496519]: Waiting for error group managing segments... Sep 22 02:20:00.704339253 (564h11m36.057124361s) [noise=true , address=45.91.116.8:4047 , piece_id=VA5OXAVATNGIPOZ4Z7V2SBCANJSG4YU5N4DWUZ27I5TOVPMWYB3Q, seg_pos={0 0}, upload=496519]: Done uploading piece. err: <nil> Sep 22 02:20:00.704633887 (564h11m36.057418985s) [noise=true , address=45.91.116.14:4190 , piece_id=ALAIMRP6VXRIJK5MXQINBNR4PIJAWR3K5DWGC6VKYYYANMFJ7UMQ, seg_pos={0 0}, upload=496519]: Done uploading piece. err: <nil> Sep 22 02:20:00.704835375 (564h11m36.057620483s) [noise=true , address=45.91.116.13:4154 , piece_id=UN7J7A65OPTBILWIDOOTMWTAWXO2BKSINOL7FQ53WSTVBJEE45KQ, seg_pos={0 0}, upload=496519]: Done uploading piece. err: <nil> Sep 22 02:20:00.704994984 (564h11m36.057780092s) [noise=true , address=45.91.116.11:4117 , piece_id=7W5B2UPMJDZZKHSOYTGGDXJ32X7AF25C6OOXAL4Y2YYP2R2JGUZQ, seg_pos={0 0}, upload=496519]: Done uploading piece. err: <nil> Sep 22 02:20:00.705242779 (564h11m36.058027887s) [noise=true , address=45.91.116.12:4226 , piece_id=5W3UXRF7Y7PDAYYZ7UDVKM4VU6UHBIJWFDFOMNR5RNNTZPDYUEQQ, seg_pos={0 0}, upload=496519]: Done uploading piece. err: <nil> Sep 22 02:20:00.705328820 (564h11m36.058113928s) [noise=true , address=45.91.116.13:4153 , piece_id=ZFD3QMFUQIIX74ZJYYHUUQFO6433AH2ESO6Y6W65SRHSDFZR5QAQ, seg_pos={0 0}, upload=496519]: Done uploading piece. err: <nil> Sep 22 02:20:00.705605951 (564h11m36.058391049s) [noise=true , address=45.91.116.12:4227 , piece_id=RBX5EOAUEONFVGFUVUYYFCKORPEAW4ZSVXAXT5X465BC2VOGMQ7Q, seg_pos={0 0}, upload=496519]: Done uploading piece. err: <nil> Sep 22 02:20:00.705704075 (564h11m36.058489183s) [noise=true , address=77.90.31.15:4699 , piece_id=2UD5RG27NTFJF2ATOHXIGB53QKMN2DUJYHCTQFDHIBJVER5WZHYQ, seg_pos={0 0}, upload=496519]: Done uploading piece. err: <nil> Sep 22 02:20:00.705994099 (564h11m36.058779207s) [noise=true , address=45.91.116.9:4262 , piece_id=Y2YMKMX3KTRUX5DL4H3LG5BXQ4FGPRBISWCIIMEEB273OXUOCK2Q, seg_pos={0 0}, upload=496519]: Done uploading piece. err: <nil> Sep 22 02:20:00.706022643 (564h11m36.058807751s) [noise=true , address=77.90.31.19:4842 , piece_id=YZN7TNH7UA4RHXEQ3PWL6ZUHHKCIXHODLWWKUKJNHJDN7A3FK4WA, seg_pos={0 0}, upload=496519]: Done uploading piece. err: <nil> Sep 22 02:20:00.706358454 (564h11m36.059143551s) [noise=true , address=45.91.116.7:4009 , piece_id=OA6AV7X76Z3G3ZV2EYLZSYAWZZTPD72H4FX2NK7NLYH6DSOKDOUA, seg_pos={0 0}, upload=496519]: Done uploading piece. err: <nil> Sep 22 02:20:00.706545063 (564h11m36.059330162s) [noise=true , address=45.91.116.8:4048 , piece_id=ACTWP4FJPDW2GXEWBRW6CXCNK7W2YQ5NMEBQR6SURCGDYB6OHOIQ, seg_pos={0 0}, upload=496519]: Done uploading piece. err: <nil> Sep 22 02:20:30.577767801 (564h12m5.9305529s) [noise=true , address=145.79.227.13:4801 , piece_id=5G6C45PY72GZZOBN7CODUFQ474VTNC5U6QIQVTPJNZKHZYOGKVHQ, seg_pos={0 0}, upload=496519]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: rpc: dial tcp 145.79.227.13:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:20:30.577816704 (564h12m5.930601841s) [noise=true , address=145.79.227.18:4006 , piece_id=XSOO52PBFUPTUK4OQD3K2WZSL5F57LPYREVFYF72EX2RCZVL62BQ, seg_pos={0 0}, upload=496519]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:20:30.577872568 (564h12m5.930657676s) [noise=true , address=145.79.227.11:4729 , piece_id=HRIXDZ7WATCOJCMUJ74JVWTE3YKDCJRYQHMQUSRGXJRX7GJRTCYQ, seg_pos={0 0}, upload=496519]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:20:30.577907233 (564h12m5.930692371s) [noise=true , address=145.79.227.5:4222 , piece_id=7MIKE4DN7D554ROEHQFOZFRWATHGQWRVTCWIPZ7X42BVC6YMOZ2A, seg_pos={0 0}, upload=496519]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:20:30.577990379 (564h12m5.930775507s) [seg_pos={0 0}, upload=496519]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:20:30.594027343 (564h12m5.946812472s) [upload=496519]: Done uploading segments... Sep 22 02:20:30.594065766 (564h12m5.946850893s) [upload=496519]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.01910409s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................................X.X.........X...] duration: 391.502649ms FAIL (2025-09-22 02:19:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 391.532175ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [............................................X.XXXXXX..X.....] duration: 534.210671ms FAIL (2025-09-22 02:19:25 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== m30.954857157s) [seg_pos={0 0}, upload=496461]: Begin upload segment... Sep 22 02:15:55.602116061 (564h7m30.954901189s) [seg_pos={0 0}, upload=496461]: Done begin upload segment. Sep 22 02:15:55.602137471 (564h7m30.954922599s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602154613 (564h7m30.954939742s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602716609 (564h7m30.955501736s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602736295 (564h7m30.955521423s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603134873 (564h7m30.955920001s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603640462 (564h7m30.95642559s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604036745 (564h7m30.956821874s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604397323 (564h7m30.95718245s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604828131 (564h7m30.957613268s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605250674 (564h7m30.958035802s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605663548 (564h7m30.958448676s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606022221 (564h7m30.958807349s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606387297 (564h7m30.959172425s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606869060 (564h7m30.959654198s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607319626 (564h7m30.960104755s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607717662 (564h7m30.960502801s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608211139 (564h7m30.960996268s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608652857 (564h7m30.961437986s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.704831022 (564h7m31.05761613s) [upload=496461]: Next returned nil segment Sep 22 02:15:55.704846542 (564h7m31.05763165s) [upload=496461]: Waiting for error group managing segments... Sep 22 02:15:55.712709156 (564h7m31.065494285s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.713081926 (564h7m31.065867054s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.716351481 (564h7m31.06913661s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717069920 (564h7m31.069855048s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717303427 (564h7m31.070088556s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725190529 (564h7m31.077975657s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725273014 (564h7m31.078058142s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732220611 (564h7m31.085005749s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732337590 (564h7m31.085122719s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.734847580 (564h7m31.087632718s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.738226892 (564h7m31.09101203s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:16:55.564886089 (564h8m30.917671218s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1AowWwVvxzmnW5qwbqbMBVbvNoigffjAN51LQp8e6XA9CZkeXL, address:145.79.227.12:4790): protocol: rpc: dial tcp 145.79.227.12:4790: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564944549 (564h8m30.917729677s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564994583 (564h8m30.917779711s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565058042 (564h8m30.91784317s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1zywsXJnw5gfpUu7mC8p7UE7ev5HexPu5nSpUH1x2pzar99T97, address:145.79.227.8:4349): protocol: rpc: dial tcp 145.79.227.8:4349: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565089361 (564h8m30.917874489s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:145.79.227.14:4847): protocol: rpc: dial tcp 145.79.227.14:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565111382 (564h8m30.91789651s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12Pxos2xcjHwcYyQnXeRL3PL72G9pXaAnBs8XnkVFkxegDRPrqp, address:145.79.227.2:4127): protocol: rpc: dial tcp 145.79.227.2:4127: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565129266 (564h8m30.917914394s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12gs9LRE3oUp1jupCw5uDvRtkNcahbq2d4VTYQsdKr6atk6vGJx, address:145.79.227.19:4095): protocol: rpc: dial tcp 145.79.227.19:4095: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565148231 (564h8m30.917933359s) [seg_pos={0 0}, upload=496461]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:16:55.580701268 (564h8m30.933486366s) [upload=496461]: Done uploading segments... Sep 22 02:16:55.580714373 (564h8m30.93349947s) [upload=496461]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 534.228896ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..........................................XXX....XXX.XXXXX..] duration: 395.89874ms FAIL (2025-09-22 02:18:57 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 02:17:57.673580556 (564h9m33.026365664s) [upload=496490]: Starting upload Sep 22 02:17:57.673647281 (564h9m33.026432389s) [upload=496490]: Done starting upload Sep 22 02:17:57.673743923 (564h9m33.026529031s) [upload=496490]: Uploading segments... Sep 22 02:17:57.673885770 (564h9m33.026670868s) [upload=496490]: Got next segment. Inline: false Sep 22 02:17:57.673917189 (564h9m33.026702287s) [upload=496490]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:17:57.673920895 (564h9m33.026705993s) [upload=496490]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:17:57.698413227 (564h9m33.051198325s) [seg_pos={0 0}, upload=496490]: Begin upload segment... Sep 22 02:17:57.698457290 (564h9m33.051242398s) [seg_pos={0 0}, upload=496490]: Done begin upload segment. Sep 22 02:17:57.698481916 (564h9m33.051267014s) [noise=true , address=145.79.227.4:4187 , piece_id=ECMWTZM4MSWM4B3QQPZDSXUHNOOF5NAILYKFVC3S4XXP7WLMCM4Q, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.698504408 (564h9m33.051289516s) [noise=true , address=45.91.116.4:4938 , piece_id=K4727TAH5CAYJ7NYWIGXGYFAWE3ACOQIC7Q77Z6AENRO7XXK4XRA, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.698816864 (564h9m33.051601962s) [noise=true , address=145.79.227.18:4005 , piece_id=KHKKCN4U4AENGSV73XRAPKKQWBS2MX3AA4SLHDELSJTP65Z34A6A, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.698828526 (564h9m33.051613634s) [noise=true , address=77.90.31.19:4841 , piece_id=NIDYBFH2UBSVILKRF3UPYBHH444BKDVK2IURJPMZJVBUGV63MWUA, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.699066383 (564h9m33.051851481s) [noise=true , address=45.91.116.8:4047 , piece_id=TMVPVMAUOVUSN4K2K5CTRLUJEAD2M37UUFCPUYCROPTGJJKOPDGQ, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.699075470 (564h9m33.051860568s) [noise=true , address=77.90.31.21:4879 , piece_id=XBXV4CX2FDGDDPWJBPFKOY55ZFEETLY2OQSW4DA6AE7SIBFPRMQQ, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.699333133 (564h9m33.052118241s) [noise=true , address=145.79.227.16:4912 , piece_id=4P6YISX6VAX7ERK3Q4XXMIZKOV7GV2LB767WOKERGIT3R2UUINIQ, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.699339675 (564h9m33.052124774s) [noise=true , address=145.79.227.2:4113 , piece_id=L4J6SOIJLASUIH4EQ74H77E66JZEM7P6EGKALL3FRWNIYI2OCOHA, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.699599293 (564h9m33.05238439s) [noise=true , address=45.91.116.6:4975 , piece_id=R6AQA2JYDB6JDPC3TQTKV4FQ2MNFY5JJIJ5NZBL4FJ273NI5N76A, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.699612266 (564h9m33.052397365s) [noise=true , address=145.79.227.4:4186 , piece_id=FIGZWR3WZREDMAQRBYWWESZT5J3CZSATMQROYT32JQUJLV6LMPEA, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.699835977 (564h9m33.052621085s) [noise=true , address=77.90.31.15:4699 , piece_id=UEOU5OX5RH4X2WFF4S6QJTHR4637U5QWPQQLDHZNFHVWFVGNDFHQ, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.699856194 (564h9m33.052641302s) [noise=true , address=45.91.116.12:4225 , piece_id=2RJQTMZ4URFUXH6AUYMEY7QUSSHQA6HZXOJXOEUEVJQFM5GDDL2Q, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.700118998 (564h9m33.052904106s) [noise=true , address=77.90.31.14:4661 , piece_id=ZTAVQJ4DHY4JAR34WWOBVPXELPHJ4NNQWEXK6NCOKTHM5UCYFDIQ, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.700353989 (564h9m33.053139097s) [noise=true , address=45.91.116.10:4081 , piece_id=2AOD7FHYIVKAUFWP5EE3O4SVXJGOCT4ZUGTXKO6XGJOKTASGQV3A, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.700862322 (564h9m33.05364743s) [noise=true , address=77.90.31.18:4807 , piece_id=3WQKTN4SRV3YJUDPMTEXQMTL4N6JDEIL3WTUOCUWKQ7HQ6YXIHQQ, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.702900216 (564h9m33.055685325s) [noise=true , address=145.79.227.17:4970 , piece_id=H3F356INVGQA7J5O225OBH7OVE55NSQ54VOU5RIOCWH2KPADQUMQ, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.780475952 (564h9m33.13326106s) [upload=496490]: Next returned nil segment Sep 22 02:17:57.780488887 (564h9m33.133273995s) [upload=496490]: Waiting for error group managing segments... Sep 22 02:17:57.788899039 (564h9m33.141684137s) [noise=true , address=45.91.116.6:4975 , piece_id=R6AQA2JYDB6JDPC3TQTKV4FQ2MNFY5JJIJ5NZBL4FJ273NI5N76A, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.788944534 (564h9m33.141729632s) [noise=true , address=45.91.116.10:4081 , piece_id=2AOD7FHYIVKAUFWP5EE3O4SVXJGOCT4ZUGTXKO6XGJOKTASGQV3A, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.790580725 (564h9m33.143365833s) [noise=true , address=45.91.116.4:4938 , piece_id=K4727TAH5CAYJ7NYWIGXGYFAWE3ACOQIC7Q77Z6AENRO7XXK4XRA, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.791180460 (564h9m33.143965568s) [noise=true , address=45.91.116.8:4047 , piece_id=TMVPVMAUOVUSN4K2K5CTRLUJEAD2M37UUFCPUYCROPTGJJKOPDGQ, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.791541127 (564h9m33.144326235s) [noise=true , address=45.91.116.12:4225 , piece_id=2RJQTMZ4URFUXH6AUYMEY7QUSSHQA6HZXOJXOEUEVJQFM5GDDL2Q, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.792519202 (564h9m33.145304301s) [noise=true , address=77.90.31.19:4841 , piece_id=NIDYBFH2UBSVILKRF3UPYBHH444BKDVK2IURJPMZJVBUGV63MWUA, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.792744376 (564h9m33.145529484s) [noise=true , address=77.90.31.14:4661 , piece_id=ZTAVQJ4DHY4JAR34WWOBVPXELPHJ4NNQWEXK6NCOKTHM5UCYFDIQ, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.792813866 (564h9m33.145598974s) [noise=true , address=77.90.31.18:4807 , piece_id=3WQKTN4SRV3YJUDPMTEXQMTL4N6JDEIL3WTUOCUWKQ7HQ6YXIHQQ, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.793326878 (564h9m33.146111986s) [noise=true , address=77.90.31.15:4699 , piece_id=UEOU5OX5RH4X2WFF4S6QJTHR4637U5QWPQQLDHZNFHVWFVGNDFHQ, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.794586813 (564h9m33.147371911s) [noise=true , address=77.90.31.21:4879 , piece_id=XBXV4CX2FDGDDPWJBPFKOY55ZFEETLY2OQSW4DA6AE7SIBFPRMQQ, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:18:27.662832742 (564h10m3.01561786s) [noise=true , address=145.79.227.2:4113 , piece_id=L4J6SOIJLASUIH4EQ74H77E66JZEM7P6EGKALL3FRWNIYI2OCOHA, seg_pos={0 0}, upload=496490]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: rpc: dial tcp 145.79.227.2:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:18:27.662897734 (564h10m3.015682852s) [noise=true , address=145.79.227.18:4005 , piece_id=KHKKCN4U4AENGSV73XRAPKKQWBS2MX3AA4SLHDELSJTP65Z34A6A, seg_pos={0 0}, upload=496490]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:18:27.662945444 (564h10m3.015730563s) [noise=true , address=145.79.227.4:4186 , piece_id=FIGZWR3WZREDMAQRBYWWESZT5J3CZSATMQROYT32JQUJLV6LMPEA, seg_pos={0 0}, upload=496490]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:18:27.662973818 (564h10m3.015758946s) [noise=true , address=145.79.227.16:4912 , piece_id=4P6YISX6VAX7ERK3Q4XXMIZKOV7GV2LB767WOKERGIT3R2UUINIQ, seg_pos={0 0}, upload=496490]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:18:27.663012731 (564h10m3.015797849s) [noise=true , address=145.79.227.4:4187 , piece_id=ECMWTZM4MSWM4B3QQPZDSXUHNOOF5NAILYKFVC3S4XXP7WLMCM4Q, seg_pos={0 0}, upload=496490]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:18:27.663027098 (564h10m3.015812226s) [noise=true , address=145.79.227.17:4970 , piece_id=H3F356INVGQA7J5O225OBH7OVE55NSQ54VOU5RIOCWH2KPADQUMQ, seg_pos={0 0}, upload=496490]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:18:27.663076862 (564h10m3.015861989s) [seg_pos={0 0}, upload=496490]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 02:18:27.681010375 (564h10m3.033795473s) [upload=496490]: Done uploading segments... Sep 22 02:18:27.681020243 (564h10m3.033805341s) [upload=496490]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 395.922414ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............................................X.X.........X..] duration: 397.122767ms FAIL (2025-09-22 02:18:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 397.136191ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...........................................XXX....XXX.XXXXX.] duration: 655.988286ms FAIL (2025-09-22 02:18:38 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 02:17:57.673580556 (564h9m33.026365664s) [upload=496490]: Starting upload Sep 22 02:17:57.673647281 (564h9m33.026432389s) [upload=496490]: Done starting upload Sep 22 02:17:57.673743923 (564h9m33.026529031s) [upload=496490]: Uploading segments... Sep 22 02:17:57.673885770 (564h9m33.026670868s) [upload=496490]: Got next segment. Inline: false Sep 22 02:17:57.673917189 (564h9m33.026702287s) [upload=496490]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:17:57.673920895 (564h9m33.026705993s) [upload=496490]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:17:57.698413227 (564h9m33.051198325s) [seg_pos={0 0}, upload=496490]: Begin upload segment... Sep 22 02:17:57.698457290 (564h9m33.051242398s) [seg_pos={0 0}, upload=496490]: Done begin upload segment. Sep 22 02:17:57.698481916 (564h9m33.051267014s) [noise=true , address=145.79.227.4:4187 , piece_id=ECMWTZM4MSWM4B3QQPZDSXUHNOOF5NAILYKFVC3S4XXP7WLMCM4Q, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.698504408 (564h9m33.051289516s) [noise=true , address=45.91.116.4:4938 , piece_id=K4727TAH5CAYJ7NYWIGXGYFAWE3ACOQIC7Q77Z6AENRO7XXK4XRA, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.698816864 (564h9m33.051601962s) [noise=true , address=145.79.227.18:4005 , piece_id=KHKKCN4U4AENGSV73XRAPKKQWBS2MX3AA4SLHDELSJTP65Z34A6A, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.698828526 (564h9m33.051613634s) [noise=true , address=77.90.31.19:4841 , piece_id=NIDYBFH2UBSVILKRF3UPYBHH444BKDVK2IURJPMZJVBUGV63MWUA, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.699066383 (564h9m33.051851481s) [noise=true , address=45.91.116.8:4047 , piece_id=TMVPVMAUOVUSN4K2K5CTRLUJEAD2M37UUFCPUYCROPTGJJKOPDGQ, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.699075470 (564h9m33.051860568s) [noise=true , address=77.90.31.21:4879 , piece_id=XBXV4CX2FDGDDPWJBPFKOY55ZFEETLY2OQSW4DA6AE7SIBFPRMQQ, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.699333133 (564h9m33.052118241s) [noise=true , address=145.79.227.16:4912 , piece_id=4P6YISX6VAX7ERK3Q4XXMIZKOV7GV2LB767WOKERGIT3R2UUINIQ, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.699339675 (564h9m33.052124774s) [noise=true , address=145.79.227.2:4113 , piece_id=L4J6SOIJLASUIH4EQ74H77E66JZEM7P6EGKALL3FRWNIYI2OCOHA, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.699599293 (564h9m33.05238439s) [noise=true , address=45.91.116.6:4975 , piece_id=R6AQA2JYDB6JDPC3TQTKV4FQ2MNFY5JJIJ5NZBL4FJ273NI5N76A, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.699612266 (564h9m33.052397365s) [noise=true , address=145.79.227.4:4186 , piece_id=FIGZWR3WZREDMAQRBYWWESZT5J3CZSATMQROYT32JQUJLV6LMPEA, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.699835977 (564h9m33.052621085s) [noise=true , address=77.90.31.15:4699 , piece_id=UEOU5OX5RH4X2WFF4S6QJTHR4637U5QWPQQLDHZNFHVWFVGNDFHQ, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.699856194 (564h9m33.052641302s) [noise=true , address=45.91.116.12:4225 , piece_id=2RJQTMZ4URFUXH6AUYMEY7QUSSHQA6HZXOJXOEUEVJQFM5GDDL2Q, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.700118998 (564h9m33.052904106s) [noise=true , address=77.90.31.14:4661 , piece_id=ZTAVQJ4DHY4JAR34WWOBVPXELPHJ4NNQWEXK6NCOKTHM5UCYFDIQ, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.700353989 (564h9m33.053139097s) [noise=true , address=45.91.116.10:4081 , piece_id=2AOD7FHYIVKAUFWP5EE3O4SVXJGOCT4ZUGTXKO6XGJOKTASGQV3A, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.700862322 (564h9m33.05364743s) [noise=true , address=77.90.31.18:4807 , piece_id=3WQKTN4SRV3YJUDPMTEXQMTL4N6JDEIL3WTUOCUWKQ7HQ6YXIHQQ, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.702900216 (564h9m33.055685325s) [noise=true , address=145.79.227.17:4970 , piece_id=H3F356INVGQA7J5O225OBH7OVE55NSQ54VOU5RIOCWH2KPADQUMQ, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.780475952 (564h9m33.13326106s) [upload=496490]: Next returned nil segment Sep 22 02:17:57.780488887 (564h9m33.133273995s) [upload=496490]: Waiting for error group managing segments... Sep 22 02:17:57.788899039 (564h9m33.141684137s) [noise=true , address=45.91.116.6:4975 , piece_id=R6AQA2JYDB6JDPC3TQTKV4FQ2MNFY5JJIJ5NZBL4FJ273NI5N76A, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.788944534 (564h9m33.141729632s) [noise=true , address=45.91.116.10:4081 , piece_id=2AOD7FHYIVKAUFWP5EE3O4SVXJGOCT4ZUGTXKO6XGJOKTASGQV3A, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.790580725 (564h9m33.143365833s) [noise=true , address=45.91.116.4:4938 , piece_id=K4727TAH5CAYJ7NYWIGXGYFAWE3ACOQIC7Q77Z6AENRO7XXK4XRA, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.791180460 (564h9m33.143965568s) [noise=true , address=45.91.116.8:4047 , piece_id=TMVPVMAUOVUSN4K2K5CTRLUJEAD2M37UUFCPUYCROPTGJJKOPDGQ, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.791541127 (564h9m33.144326235s) [noise=true , address=45.91.116.12:4225 , piece_id=2RJQTMZ4URFUXH6AUYMEY7QUSSHQA6HZXOJXOEUEVJQFM5GDDL2Q, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.792519202 (564h9m33.145304301s) [noise=true , address=77.90.31.19:4841 , piece_id=NIDYBFH2UBSVILKRF3UPYBHH444BKDVK2IURJPMZJVBUGV63MWUA, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.792744376 (564h9m33.145529484s) [noise=true , address=77.90.31.14:4661 , piece_id=ZTAVQJ4DHY4JAR34WWOBVPXELPHJ4NNQWEXK6NCOKTHM5UCYFDIQ, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.792813866 (564h9m33.145598974s) [noise=true , address=77.90.31.18:4807 , piece_id=3WQKTN4SRV3YJUDPMTEXQMTL4N6JDEIL3WTUOCUWKQ7HQ6YXIHQQ, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.793326878 (564h9m33.146111986s) [noise=true , address=77.90.31.15:4699 , piece_id=UEOU5OX5RH4X2WFF4S6QJTHR4637U5QWPQQLDHZNFHVWFVGNDFHQ, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.794586813 (564h9m33.147371911s) [noise=true , address=77.90.31.21:4879 , piece_id=XBXV4CX2FDGDDPWJBPFKOY55ZFEETLY2OQSW4DA6AE7SIBFPRMQQ, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:18:27.662832742 (564h10m3.01561786s) [noise=true , address=145.79.227.2:4113 , piece_id=L4J6SOIJLASUIH4EQ74H77E66JZEM7P6EGKALL3FRWNIYI2OCOHA, seg_pos={0 0}, upload=496490]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: rpc: dial tcp 145.79.227.2:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:18:27.662897734 (564h10m3.015682852s) [noise=true , address=145.79.227.18:4005 , piece_id=KHKKCN4U4AENGSV73XRAPKKQWBS2MX3AA4SLHDELSJTP65Z34A6A, seg_pos={0 0}, upload=496490]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:18:27.662945444 (564h10m3.015730563s) [noise=true , address=145.79.227.4:4186 , piece_id=FIGZWR3WZREDMAQRBYWWESZT5J3CZSATMQROYT32JQUJLV6LMPEA, seg_pos={0 0}, upload=496490]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:18:27.662973818 (564h10m3.015758946s) [noise=true , address=145.79.227.16:4912 , piece_id=4P6YISX6VAX7ERK3Q4XXMIZKOV7GV2LB767WOKERGIT3R2UUINIQ, seg_pos={0 0}, upload=496490]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:18:27.663012731 (564h10m3.015797849s) [noise=true , address=145.79.227.4:4187 , piece_id=ECMWTZM4MSWM4B3QQPZDSXUHNOOF5NAILYKFVC3S4XXP7WLMCM4Q, seg_pos={0 0}, upload=496490]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:18:27.663027098 (564h10m3.015812226s) [noise=true , address=145.79.227.17:4970 , piece_id=H3F356INVGQA7J5O225OBH7OVE55NSQ54VOU5RIOCWH2KPADQUMQ, seg_pos={0 0}, upload=496490]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:18:27.663076862 (564h10m3.015861989s) [seg_pos={0 0}, upload=496490]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 02:18:27.681010375 (564h10m3.033795473s) [upload=496490]: Done uploading segments... Sep 22 02:18:27.681020243 (564h10m3.033805341s) [upload=496490]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 656.018462ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [.............................................X.XXXXXX..X....] duration: 511.78548ms FAIL (2025-09-22 02:18:38 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== m30.954857157s) [seg_pos={0 0}, upload=496461]: Begin upload segment... Sep 22 02:15:55.602116061 (564h7m30.954901189s) [seg_pos={0 0}, upload=496461]: Done begin upload segment. Sep 22 02:15:55.602137471 (564h7m30.954922599s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602154613 (564h7m30.954939742s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602716609 (564h7m30.955501736s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602736295 (564h7m30.955521423s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603134873 (564h7m30.955920001s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603640462 (564h7m30.95642559s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604036745 (564h7m30.956821874s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604397323 (564h7m30.95718245s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604828131 (564h7m30.957613268s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605250674 (564h7m30.958035802s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605663548 (564h7m30.958448676s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606022221 (564h7m30.958807349s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606387297 (564h7m30.959172425s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606869060 (564h7m30.959654198s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607319626 (564h7m30.960104755s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607717662 (564h7m30.960502801s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608211139 (564h7m30.960996268s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608652857 (564h7m30.961437986s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.704831022 (564h7m31.05761613s) [upload=496461]: Next returned nil segment Sep 22 02:15:55.704846542 (564h7m31.05763165s) [upload=496461]: Waiting for error group managing segments... Sep 22 02:15:55.712709156 (564h7m31.065494285s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.713081926 (564h7m31.065867054s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.716351481 (564h7m31.06913661s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717069920 (564h7m31.069855048s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717303427 (564h7m31.070088556s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725190529 (564h7m31.077975657s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725273014 (564h7m31.078058142s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732220611 (564h7m31.085005749s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732337590 (564h7m31.085122719s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.734847580 (564h7m31.087632718s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.738226892 (564h7m31.09101203s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:16:55.564886089 (564h8m30.917671218s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1AowWwVvxzmnW5qwbqbMBVbvNoigffjAN51LQp8e6XA9CZkeXL, address:145.79.227.12:4790): protocol: rpc: dial tcp 145.79.227.12:4790: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564944549 (564h8m30.917729677s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564994583 (564h8m30.917779711s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565058042 (564h8m30.91784317s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1zywsXJnw5gfpUu7mC8p7UE7ev5HexPu5nSpUH1x2pzar99T97, address:145.79.227.8:4349): protocol: rpc: dial tcp 145.79.227.8:4349: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565089361 (564h8m30.917874489s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:145.79.227.14:4847): protocol: rpc: dial tcp 145.79.227.14:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565111382 (564h8m30.91789651s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12Pxos2xcjHwcYyQnXeRL3PL72G9pXaAnBs8XnkVFkxegDRPrqp, address:145.79.227.2:4127): protocol: rpc: dial tcp 145.79.227.2:4127: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565129266 (564h8m30.917914394s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12gs9LRE3oUp1jupCw5uDvRtkNcahbq2d4VTYQsdKr6atk6vGJx, address:145.79.227.19:4095): protocol: rpc: dial tcp 145.79.227.19:4095: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565148231 (564h8m30.917933359s) [seg_pos={0 0}, upload=496461]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:16:55.580701268 (564h8m30.933486366s) [upload=496461]: Done uploading segments... Sep 22 02:16:55.580714373 (564h8m30.93349947s) [upload=496461]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 511.803132ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............................................X.X.........X.] duration: 472.620203ms FAIL (2025-09-22 02:18:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 472.640953ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [............................................XXX....XXX.XXXXX] duration: 30.019638448s FAIL (2025-09-22 02:18:27 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 02:17:57.673580556 (564h9m33.026365664s) [upload=496490]: Starting upload Sep 22 02:17:57.673647281 (564h9m33.026432389s) [upload=496490]: Done starting upload Sep 22 02:17:57.673743923 (564h9m33.026529031s) [upload=496490]: Uploading segments... Sep 22 02:17:57.673885770 (564h9m33.026670868s) [upload=496490]: Got next segment. Inline: false Sep 22 02:17:57.673917189 (564h9m33.026702287s) [upload=496490]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:17:57.673920895 (564h9m33.026705993s) [upload=496490]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:17:57.698413227 (564h9m33.051198325s) [seg_pos={0 0}, upload=496490]: Begin upload segment... Sep 22 02:17:57.698457290 (564h9m33.051242398s) [seg_pos={0 0}, upload=496490]: Done begin upload segment. Sep 22 02:17:57.698481916 (564h9m33.051267014s) [noise=true , address=145.79.227.4:4187 , piece_id=ECMWTZM4MSWM4B3QQPZDSXUHNOOF5NAILYKFVC3S4XXP7WLMCM4Q, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.698504408 (564h9m33.051289516s) [noise=true , address=45.91.116.4:4938 , piece_id=K4727TAH5CAYJ7NYWIGXGYFAWE3ACOQIC7Q77Z6AENRO7XXK4XRA, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.698816864 (564h9m33.051601962s) [noise=true , address=145.79.227.18:4005 , piece_id=KHKKCN4U4AENGSV73XRAPKKQWBS2MX3AA4SLHDELSJTP65Z34A6A, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.698828526 (564h9m33.051613634s) [noise=true , address=77.90.31.19:4841 , piece_id=NIDYBFH2UBSVILKRF3UPYBHH444BKDVK2IURJPMZJVBUGV63MWUA, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.699066383 (564h9m33.051851481s) [noise=true , address=45.91.116.8:4047 , piece_id=TMVPVMAUOVUSN4K2K5CTRLUJEAD2M37UUFCPUYCROPTGJJKOPDGQ, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.699075470 (564h9m33.051860568s) [noise=true , address=77.90.31.21:4879 , piece_id=XBXV4CX2FDGDDPWJBPFKOY55ZFEETLY2OQSW4DA6AE7SIBFPRMQQ, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.699333133 (564h9m33.052118241s) [noise=true , address=145.79.227.16:4912 , piece_id=4P6YISX6VAX7ERK3Q4XXMIZKOV7GV2LB767WOKERGIT3R2UUINIQ, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.699339675 (564h9m33.052124774s) [noise=true , address=145.79.227.2:4113 , piece_id=L4J6SOIJLASUIH4EQ74H77E66JZEM7P6EGKALL3FRWNIYI2OCOHA, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.699599293 (564h9m33.05238439s) [noise=true , address=45.91.116.6:4975 , piece_id=R6AQA2JYDB6JDPC3TQTKV4FQ2MNFY5JJIJ5NZBL4FJ273NI5N76A, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.699612266 (564h9m33.052397365s) [noise=true , address=145.79.227.4:4186 , piece_id=FIGZWR3WZREDMAQRBYWWESZT5J3CZSATMQROYT32JQUJLV6LMPEA, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.699835977 (564h9m33.052621085s) [noise=true , address=77.90.31.15:4699 , piece_id=UEOU5OX5RH4X2WFF4S6QJTHR4637U5QWPQQLDHZNFHVWFVGNDFHQ, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.699856194 (564h9m33.052641302s) [noise=true , address=45.91.116.12:4225 , piece_id=2RJQTMZ4URFUXH6AUYMEY7QUSSHQA6HZXOJXOEUEVJQFM5GDDL2Q, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.700118998 (564h9m33.052904106s) [noise=true , address=77.90.31.14:4661 , piece_id=ZTAVQJ4DHY4JAR34WWOBVPXELPHJ4NNQWEXK6NCOKTHM5UCYFDIQ, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.700353989 (564h9m33.053139097s) [noise=true , address=45.91.116.10:4081 , piece_id=2AOD7FHYIVKAUFWP5EE3O4SVXJGOCT4ZUGTXKO6XGJOKTASGQV3A, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.700862322 (564h9m33.05364743s) [noise=true , address=77.90.31.18:4807 , piece_id=3WQKTN4SRV3YJUDPMTEXQMTL4N6JDEIL3WTUOCUWKQ7HQ6YXIHQQ, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.702900216 (564h9m33.055685325s) [noise=true , address=145.79.227.17:4970 , piece_id=H3F356INVGQA7J5O225OBH7OVE55NSQ54VOU5RIOCWH2KPADQUMQ, seg_pos={0 0}, upload=496490]: Uploading piece... Sep 22 02:17:57.780475952 (564h9m33.13326106s) [upload=496490]: Next returned nil segment Sep 22 02:17:57.780488887 (564h9m33.133273995s) [upload=496490]: Waiting for error group managing segments... Sep 22 02:17:57.788899039 (564h9m33.141684137s) [noise=true , address=45.91.116.6:4975 , piece_id=R6AQA2JYDB6JDPC3TQTKV4FQ2MNFY5JJIJ5NZBL4FJ273NI5N76A, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.788944534 (564h9m33.141729632s) [noise=true , address=45.91.116.10:4081 , piece_id=2AOD7FHYIVKAUFWP5EE3O4SVXJGOCT4ZUGTXKO6XGJOKTASGQV3A, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.790580725 (564h9m33.143365833s) [noise=true , address=45.91.116.4:4938 , piece_id=K4727TAH5CAYJ7NYWIGXGYFAWE3ACOQIC7Q77Z6AENRO7XXK4XRA, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.791180460 (564h9m33.143965568s) [noise=true , address=45.91.116.8:4047 , piece_id=TMVPVMAUOVUSN4K2K5CTRLUJEAD2M37UUFCPUYCROPTGJJKOPDGQ, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.791541127 (564h9m33.144326235s) [noise=true , address=45.91.116.12:4225 , piece_id=2RJQTMZ4URFUXH6AUYMEY7QUSSHQA6HZXOJXOEUEVJQFM5GDDL2Q, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.792519202 (564h9m33.145304301s) [noise=true , address=77.90.31.19:4841 , piece_id=NIDYBFH2UBSVILKRF3UPYBHH444BKDVK2IURJPMZJVBUGV63MWUA, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.792744376 (564h9m33.145529484s) [noise=true , address=77.90.31.14:4661 , piece_id=ZTAVQJ4DHY4JAR34WWOBVPXELPHJ4NNQWEXK6NCOKTHM5UCYFDIQ, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.792813866 (564h9m33.145598974s) [noise=true , address=77.90.31.18:4807 , piece_id=3WQKTN4SRV3YJUDPMTEXQMTL4N6JDEIL3WTUOCUWKQ7HQ6YXIHQQ, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.793326878 (564h9m33.146111986s) [noise=true , address=77.90.31.15:4699 , piece_id=UEOU5OX5RH4X2WFF4S6QJTHR4637U5QWPQQLDHZNFHVWFVGNDFHQ, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:17:57.794586813 (564h9m33.147371911s) [noise=true , address=77.90.31.21:4879 , piece_id=XBXV4CX2FDGDDPWJBPFKOY55ZFEETLY2OQSW4DA6AE7SIBFPRMQQ, seg_pos={0 0}, upload=496490]: Done uploading piece. err: <nil> Sep 22 02:18:27.662832742 (564h10m3.01561786s) [noise=true , address=145.79.227.2:4113 , piece_id=L4J6SOIJLASUIH4EQ74H77E66JZEM7P6EGKALL3FRWNIYI2OCOHA, seg_pos={0 0}, upload=496490]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: rpc: dial tcp 145.79.227.2:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:18:27.662897734 (564h10m3.015682852s) [noise=true , address=145.79.227.18:4005 , piece_id=KHKKCN4U4AENGSV73XRAPKKQWBS2MX3AA4SLHDELSJTP65Z34A6A, seg_pos={0 0}, upload=496490]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:18:27.662945444 (564h10m3.015730563s) [noise=true , address=145.79.227.4:4186 , piece_id=FIGZWR3WZREDMAQRBYWWESZT5J3CZSATMQROYT32JQUJLV6LMPEA, seg_pos={0 0}, upload=496490]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:18:27.662973818 (564h10m3.015758946s) [noise=true , address=145.79.227.16:4912 , piece_id=4P6YISX6VAX7ERK3Q4XXMIZKOV7GV2LB767WOKERGIT3R2UUINIQ, seg_pos={0 0}, upload=496490]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:18:27.663012731 (564h10m3.015797849s) [noise=true , address=145.79.227.4:4187 , piece_id=ECMWTZM4MSWM4B3QQPZDSXUHNOOF5NAILYKFVC3S4XXP7WLMCM4Q, seg_pos={0 0}, upload=496490]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:18:27.663027098 (564h10m3.015812226s) [noise=true , address=145.79.227.17:4970 , piece_id=H3F356INVGQA7J5O225OBH7OVE55NSQ54VOU5RIOCWH2KPADQUMQ, seg_pos={0 0}, upload=496490]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:18:27.663076862 (564h10m3.015861989s) [seg_pos={0 0}, upload=496490]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 02:18:27.681010375 (564h10m3.033795473s) [upload=496490]: Done uploading segments... Sep 22 02:18:27.681020243 (564h10m3.033805341s) [upload=496490]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.019654087s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)