Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [..............................................X.XXXXXX..X...] duration: 399.855716ms FAIL (2025-09-22 02:18:18 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== m30.954857157s) [seg_pos={0 0}, upload=496461]: Begin upload segment... Sep 22 02:15:55.602116061 (564h7m30.954901189s) [seg_pos={0 0}, upload=496461]: Done begin upload segment. Sep 22 02:15:55.602137471 (564h7m30.954922599s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602154613 (564h7m30.954939742s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602716609 (564h7m30.955501736s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602736295 (564h7m30.955521423s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603134873 (564h7m30.955920001s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603640462 (564h7m30.95642559s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604036745 (564h7m30.956821874s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604397323 (564h7m30.95718245s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604828131 (564h7m30.957613268s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605250674 (564h7m30.958035802s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605663548 (564h7m30.958448676s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606022221 (564h7m30.958807349s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606387297 (564h7m30.959172425s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606869060 (564h7m30.959654198s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607319626 (564h7m30.960104755s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607717662 (564h7m30.960502801s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608211139 (564h7m30.960996268s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608652857 (564h7m30.961437986s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.704831022 (564h7m31.05761613s) [upload=496461]: Next returned nil segment Sep 22 02:15:55.704846542 (564h7m31.05763165s) [upload=496461]: Waiting for error group managing segments... Sep 22 02:15:55.712709156 (564h7m31.065494285s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.713081926 (564h7m31.065867054s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.716351481 (564h7m31.06913661s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717069920 (564h7m31.069855048s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717303427 (564h7m31.070088556s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725190529 (564h7m31.077975657s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725273014 (564h7m31.078058142s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732220611 (564h7m31.085005749s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732337590 (564h7m31.085122719s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.734847580 (564h7m31.087632718s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.738226892 (564h7m31.09101203s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:16:55.564886089 (564h8m30.917671218s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1AowWwVvxzmnW5qwbqbMBVbvNoigffjAN51LQp8e6XA9CZkeXL, address:145.79.227.12:4790): protocol: rpc: dial tcp 145.79.227.12:4790: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564944549 (564h8m30.917729677s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564994583 (564h8m30.917779711s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565058042 (564h8m30.91784317s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1zywsXJnw5gfpUu7mC8p7UE7ev5HexPu5nSpUH1x2pzar99T97, address:145.79.227.8:4349): protocol: rpc: dial tcp 145.79.227.8:4349: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565089361 (564h8m30.917874489s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:145.79.227.14:4847): protocol: rpc: dial tcp 145.79.227.14:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565111382 (564h8m30.91789651s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12Pxos2xcjHwcYyQnXeRL3PL72G9pXaAnBs8XnkVFkxegDRPrqp, address:145.79.227.2:4127): protocol: rpc: dial tcp 145.79.227.2:4127: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565129266 (564h8m30.917914394s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12gs9LRE3oUp1jupCw5uDvRtkNcahbq2d4VTYQsdKr6atk6vGJx, address:145.79.227.19:4095): protocol: rpc: dial tcp 145.79.227.19:4095: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565148231 (564h8m30.917933359s) [seg_pos={0 0}, upload=496461]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:16:55.580701268 (564h8m30.933486366s) [upload=496461]: Done uploading segments... Sep 22 02:16:55.580714373 (564h8m30.93349947s) [upload=496461]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 399.873759ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............................................X.X.........X] duration: 30.001269716s FAIL (2025-09-22 02:17:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001311125s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.............................................XXX....XXX.XXXX] duration: 30.020429055s FAIL (2025-09-22 02:17:20 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Sep 22 02:16:50.768177348 (564h8m26.120962477s) [upload=496475]: Starting upload Sep 22 02:16:50.768255395 (564h8m26.121040523s) [upload=496475]: Done starting upload Sep 22 02:16:50.768385710 (564h8m26.121170848s) [upload=496475]: Uploading segments... Sep 22 02:16:50.768653372 (564h8m26.1214385s) [upload=496475]: Got next segment. Inline: false Sep 22 02:16:50.768711932 (564h8m26.12149706s) [upload=496475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:16:50.768720598 (564h8m26.121505726s) [upload=496475]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:16:50.790313058 (564h8m26.143098186s) [seg_pos={0 0}, upload=496475]: Begin upload segment... Sep 22 02:16:50.790349977 (564h8m26.143135105s) [seg_pos={0 0}, upload=496475]: Done begin upload segment. Sep 22 02:16:50.790377399 (564h8m26.143162526s) [noise=true , address=45.91.116.4:4940 , piece_id=J4EFLRV4FCBE3NJRBTF7KTV7VV66OUNHWBK42WUAMYXMBAHIP3FQ, seg_pos={0 0}, upload=496475]: Uploading piece... Sep 22 02:16:50.790431550 (564h8m26.143216688s) [noise=true , address=145.79.227.3:4151 , piece_id=HJDBWULPYFGPQESSVGEA4V3UWIKC4XLSFFYEJBYRYK7VBNPJLWJA, seg_pos={0 0}, upload=496475]: Uploading piece... Sep 22 02:16:50.790869753 (564h8m26.143654881s) [noise=true , address=77.90.31.12:4592 , piece_id=MC3DRY2QLJXN23F2R7ZDJLJNDQTI6H3SL4K65V6HAJMVQ5H2RASQ, seg_pos={0 0}, upload=496475]: Uploading piece... Sep 22 02:16:50.790887316 (564h8m26.143672454s) [noise=true , address=145.79.227.16:4910 , piece_id=HT53OSUQIKWEZSVT56HZST7M4NC6N3TTRM6DHDG3NI5EPDMLIZVA, seg_pos={0 0}, upload=496475]: Uploading piece... Sep 22 02:16:50.791256689 (564h8m26.144041827s) [noise=true , address=145.79.227.11:4731 , piece_id=QISNULOHPMWVZW3C2H6ZO2UFY4VYZDLMVLWB7DSP6YGBAS3DB7EQ, seg_pos={0 0}, upload=496475]: Uploading piece... Sep 22 02:16:50.791281886 (564h8m26.144067014s) [noise=true , address=77.90.31.16:4733 , piece_id=5ZEMMHO3M3Z3CFB3FLJP5TM5PXAMF7LWL2FZER6GOFGEQRFGG6QA, seg_pos={0 0}, upload=496475]: Uploading piece... Sep 22 02:16:50.791636781 (564h8m26.14442191s) [noise=true , address=145.79.227.12:4767 , piece_id=3JCDTIG2NJNLYEIMN7PZ4BI7TMBTW44OMXCPF5YBD5YFNX3ELVOA, seg_pos={0 0}, upload=496475]: Uploading piece... Sep 22 02:16:50.791737231 (564h8m26.144522369s) [noise=true , address=77.90.31.10:4520 , piece_id=BA5NGD46QOPEJLTAOZY7XPXKVYOUKMH324NMK2X24FJ5YK75IACQ, seg_pos={0 0}, upload=496475]: Uploading piece... Sep 22 02:16:50.792021404 (564h8m26.144806532s) [noise=true , address=145.79.227.19:4078 , piece_id=G5FBDBSOHLI22UU6CR7LIJ3IRJY32N2XBFVUUSCGLIMJP2LANCZA, seg_pos={0 0}, upload=496475]: Uploading piece... Sep 22 02:16:50.792143603 (564h8m26.144928732s) [noise=true , address=45.91.116.11:4117 , piece_id=D4KEIVXJ7UGNDIP46LER3SR5AJ4DX7Y2MRE4LDCOXJ37VR7RBWXA, seg_pos={0 0}, upload=496475]: Uploading piece... Sep 22 02:16:50.792378764 (564h8m26.145163893s) [noise=true , address=45.91.116.14:4191 , piece_id=TGWM2MYQQIE5ENTNPKTFBBIANYITFQSJSZXKEPI4SE3EL7MFXDNQ, seg_pos={0 0}, upload=496475]: Uploading piece... Sep 22 02:16:50.792582556 (564h8m26.145367685s) [noise=true , address=145.79.227.18:4006 , piece_id=KN2U5756TMXDAHYOH24GPBBZVONFHA4SMAFFHJ7Q4LONZ2IL7ONQ, seg_pos={0 0}, upload=496475]: Uploading piece... Sep 22 02:16:50.792754960 (564h8m26.145540089s) [noise=true , address=77.90.31.17:4771 , piece_id=B2I4Z2DCAXDUTQSORD6RMDOQRY2SPRUR6ONIL67HYCYUH6OLXXRA, seg_pos={0 0}, upload=496475]: Uploading piece... Sep 22 02:16:50.792952792 (564h8m26.14573792s) [noise=true , address=145.79.227.13:4801 , piece_id=DQZ7Q6CALOIN53RDB4MUTBHARPDUQNSA6VYWIDOXGHPDU3OAU42A, seg_pos={0 0}, upload=496475]: Uploading piece... Sep 22 02:16:50.793306515 (564h8m26.146091653s) [noise=true , address=45.91.116.6:4974 , piece_id=FEB6W6MKD4RDNEQ6D6V5S4BAT4PLGZUSI3O7FXKK66SVQOJFPPBQ, seg_pos={0 0}, upload=496475]: Uploading piece... Sep 22 02:16:50.793761109 (564h8m26.146546237s) [noise=true , address=45.91.116.11:4119 , piece_id=IWFHBANEHU66W76NCTW26Q6J2XRZVJROEFOOFKOH2F4TQONA6YOA, seg_pos={0 0}, upload=496475]: Uploading piece... Sep 22 02:16:50.898128080 (564h8m26.250913208s) [upload=496475]: Next returned nil segment Sep 22 02:16:50.898171271 (564h8m26.250956409s) [upload=496475]: Waiting for error group managing segments... Sep 22 02:16:50.908266325 (564h8m26.261051463s) [noise=true , address=45.91.116.14:4191 , piece_id=TGWM2MYQQIE5ENTNPKTFBBIANYITFQSJSZXKEPI4SE3EL7MFXDNQ, seg_pos={0 0}, upload=496475]: Done uploading piece. err: <nil> Sep 22 02:16:50.908317722 (564h8m26.261102849s) [noise=true , address=45.91.116.6:4974 , piece_id=FEB6W6MKD4RDNEQ6D6V5S4BAT4PLGZUSI3O7FXKK66SVQOJFPPBQ, seg_pos={0 0}, upload=496475]: Done uploading piece. err: <nil> Sep 22 02:16:50.908832066 (564h8m26.261617194s) [noise=true , address=45.91.116.11:4117 , piece_id=D4KEIVXJ7UGNDIP46LER3SR5AJ4DX7Y2MRE4LDCOXJ37VR7RBWXA, seg_pos={0 0}, upload=496475]: Done uploading piece. err: <nil> Sep 22 02:16:50.909128182 (564h8m26.261913311s) [noise=true , address=45.91.116.11:4119 , piece_id=IWFHBANEHU66W76NCTW26Q6J2XRZVJROEFOOFKOH2F4TQONA6YOA, seg_pos={0 0}, upload=496475]: Done uploading piece. err: <nil> Sep 22 02:16:50.909856789 (564h8m26.262641917s) [noise=true , address=77.90.31.16:4733 , piece_id=5ZEMMHO3M3Z3CFB3FLJP5TM5PXAMF7LWL2FZER6GOFGEQRFGG6QA, seg_pos={0 0}, upload=496475]: Done uploading piece. err: <nil> Sep 22 02:16:50.909934695 (564h8m26.262719823s) [noise=true , address=45.91.116.4:4940 , piece_id=J4EFLRV4FCBE3NJRBTF7KTV7VV66OUNHWBK42WUAMYXMBAHIP3FQ, seg_pos={0 0}, upload=496475]: Done uploading piece. err: <nil> Sep 22 02:16:50.910839203 (564h8m26.26362434s) [noise=true , address=77.90.31.12:4592 , piece_id=MC3DRY2QLJXN23F2R7ZDJLJNDQTI6H3SL4K65V6HAJMVQ5H2RASQ, seg_pos={0 0}, upload=496475]: Done uploading piece. err: <nil> Sep 22 02:16:50.911013350 (564h8m26.263798479s) [noise=true , address=77.90.31.17:4771 , piece_id=B2I4Z2DCAXDUTQSORD6RMDOQRY2SPRUR6ONIL67HYCYUH6OLXXRA, seg_pos={0 0}, upload=496475]: Done uploading piece. err: <nil> Sep 22 02:16:50.911570605 (564h8m26.264355734s) [noise=true , address=77.90.31.10:4520 , piece_id=BA5NGD46QOPEJLTAOZY7XPXKVYOUKMH324NMK2X24FJ5YK75IACQ, seg_pos={0 0}, upload=496475]: Done uploading piece. err: <nil> Sep 22 02:17:20.755248504 (564h8m56.108033632s) [noise=true , address=145.79.227.18:4006 , piece_id=KN2U5756TMXDAHYOH24GPBBZVONFHA4SMAFFHJ7Q4LONZ2IL7ONQ, seg_pos={0 0}, upload=496475]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: rpc: dial tcp 145.79.227.18:4006: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:20.755310430 (564h8m56.108095568s) [noise=true , address=145.79.227.3:4151 , piece_id=HJDBWULPYFGPQESSVGEA4V3UWIKC4XLSFFYEJBYRYK7VBNPJLWJA, seg_pos={0 0}, upload=496475]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:20.755531385 (564h8m56.108316513s) [noise=true , address=145.79.227.19:4078 , piece_id=G5FBDBSOHLI22UU6CR7LIJ3IRJY32N2XBFVUUSCGLIMJP2LANCZA, seg_pos={0 0}, upload=496475]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:20.755619941 (564h8m56.10840508s) [noise=true , address=145.79.227.12:4767 , piece_id=3JCDTIG2NJNLYEIMN7PZ4BI7TMBTW44OMXCPF5YBD5YFNX3ELVOA, seg_pos={0 0}, upload=496475]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: rpc: dial tcp 145.79.227.12:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:20.755635040 (564h8m56.108420168s) [noise=true , address=145.79.227.13:4801 , piece_id=DQZ7Q6CALOIN53RDB4MUTBHARPDUQNSA6VYWIDOXGHPDU3OAU42A, seg_pos={0 0}, upload=496475]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: rpc: dial tcp 145.79.227.13:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:20.755671458 (564h8m56.108456586s) [noise=true , address=145.79.227.11:4731 , piece_id=QISNULOHPMWVZW3C2H6ZO2UFY4VYZDLMVLWB7DSP6YGBAS3DB7EQ, seg_pos={0 0}, upload=496475]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:20.755720891 (564h8m56.108506019s) [noise=true , address=145.79.227.16:4910 , piece_id=HT53OSUQIKWEZSVT56HZST7M4NC6N3TTRM6DHDG3NI5EPDMLIZVA, seg_pos={0 0}, upload=496475]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:20.755753643 (564h8m56.10853877s) [seg_pos={0 0}, upload=496475]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 22 02:17:20.773420015 (564h8m56.126205143s) [upload=496475]: Done uploading segments... Sep 22 02:17:20.773461933 (564h8m56.126247062s) [upload=496475]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.020446797s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [...............................................X.XXXXXX..X..] duration: 562.242746ms FAIL (2025-09-22 02:17:15 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== m30.954857157s) [seg_pos={0 0}, upload=496461]: Begin upload segment... Sep 22 02:15:55.602116061 (564h7m30.954901189s) [seg_pos={0 0}, upload=496461]: Done begin upload segment. Sep 22 02:15:55.602137471 (564h7m30.954922599s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602154613 (564h7m30.954939742s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602716609 (564h7m30.955501736s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602736295 (564h7m30.955521423s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603134873 (564h7m30.955920001s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603640462 (564h7m30.95642559s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604036745 (564h7m30.956821874s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604397323 (564h7m30.95718245s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604828131 (564h7m30.957613268s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605250674 (564h7m30.958035802s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605663548 (564h7m30.958448676s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606022221 (564h7m30.958807349s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606387297 (564h7m30.959172425s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606869060 (564h7m30.959654198s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607319626 (564h7m30.960104755s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607717662 (564h7m30.960502801s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608211139 (564h7m30.960996268s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608652857 (564h7m30.961437986s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.704831022 (564h7m31.05761613s) [upload=496461]: Next returned nil segment Sep 22 02:15:55.704846542 (564h7m31.05763165s) [upload=496461]: Waiting for error group managing segments... Sep 22 02:15:55.712709156 (564h7m31.065494285s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.713081926 (564h7m31.065867054s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.716351481 (564h7m31.06913661s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717069920 (564h7m31.069855048s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717303427 (564h7m31.070088556s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725190529 (564h7m31.077975657s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725273014 (564h7m31.078058142s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732220611 (564h7m31.085005749s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732337590 (564h7m31.085122719s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.734847580 (564h7m31.087632718s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.738226892 (564h7m31.09101203s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:16:55.564886089 (564h8m30.917671218s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1AowWwVvxzmnW5qwbqbMBVbvNoigffjAN51LQp8e6XA9CZkeXL, address:145.79.227.12:4790): protocol: rpc: dial tcp 145.79.227.12:4790: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564944549 (564h8m30.917729677s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564994583 (564h8m30.917779711s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565058042 (564h8m30.91784317s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1zywsXJnw5gfpUu7mC8p7UE7ev5HexPu5nSpUH1x2pzar99T97, address:145.79.227.8:4349): protocol: rpc: dial tcp 145.79.227.8:4349: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565089361 (564h8m30.917874489s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:145.79.227.14:4847): protocol: rpc: dial tcp 145.79.227.14:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565111382 (564h8m30.91789651s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12Pxos2xcjHwcYyQnXeRL3PL72G9pXaAnBs8XnkVFkxegDRPrqp, address:145.79.227.2:4127): protocol: rpc: dial tcp 145.79.227.2:4127: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565129266 (564h8m30.917914394s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12gs9LRE3oUp1jupCw5uDvRtkNcahbq2d4VTYQsdKr6atk6vGJx, address:145.79.227.19:4095): protocol: rpc: dial tcp 145.79.227.19:4095: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565148231 (564h8m30.917933359s) [seg_pos={0 0}, upload=496461]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:16:55.580701268 (564h8m30.933486366s) [upload=496461]: Done uploading segments... Sep 22 02:16:55.580714373 (564h8m30.93349947s) [upload=496461]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 562.283261ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [................................................X.XXXXXX..X.] duration: 437.229431ms FAIL (2025-09-22 02:16:56 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== m30.954857157s) [seg_pos={0 0}, upload=496461]: Begin upload segment... Sep 22 02:15:55.602116061 (564h7m30.954901189s) [seg_pos={0 0}, upload=496461]: Done begin upload segment. Sep 22 02:15:55.602137471 (564h7m30.954922599s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602154613 (564h7m30.954939742s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602716609 (564h7m30.955501736s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602736295 (564h7m30.955521423s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603134873 (564h7m30.955920001s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603640462 (564h7m30.95642559s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604036745 (564h7m30.956821874s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604397323 (564h7m30.95718245s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604828131 (564h7m30.957613268s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605250674 (564h7m30.958035802s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605663548 (564h7m30.958448676s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606022221 (564h7m30.958807349s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606387297 (564h7m30.959172425s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606869060 (564h7m30.959654198s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607319626 (564h7m30.960104755s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607717662 (564h7m30.960502801s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608211139 (564h7m30.960996268s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608652857 (564h7m30.961437986s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.704831022 (564h7m31.05761613s) [upload=496461]: Next returned nil segment Sep 22 02:15:55.704846542 (564h7m31.05763165s) [upload=496461]: Waiting for error group managing segments... Sep 22 02:15:55.712709156 (564h7m31.065494285s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.713081926 (564h7m31.065867054s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.716351481 (564h7m31.06913661s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717069920 (564h7m31.069855048s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717303427 (564h7m31.070088556s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725190529 (564h7m31.077975657s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725273014 (564h7m31.078058142s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732220611 (564h7m31.085005749s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732337590 (564h7m31.085122719s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.734847580 (564h7m31.087632718s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.738226892 (564h7m31.09101203s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:16:55.564886089 (564h8m30.917671218s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1AowWwVvxzmnW5qwbqbMBVbvNoigffjAN51LQp8e6XA9CZkeXL, address:145.79.227.12:4790): protocol: rpc: dial tcp 145.79.227.12:4790: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564944549 (564h8m30.917729677s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564994583 (564h8m30.917779711s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565058042 (564h8m30.91784317s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1zywsXJnw5gfpUu7mC8p7UE7ev5HexPu5nSpUH1x2pzar99T97, address:145.79.227.8:4349): protocol: rpc: dial tcp 145.79.227.8:4349: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565089361 (564h8m30.917874489s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:145.79.227.14:4847): protocol: rpc: dial tcp 145.79.227.14:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565111382 (564h8m30.91789651s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12Pxos2xcjHwcYyQnXeRL3PL72G9pXaAnBs8XnkVFkxegDRPrqp, address:145.79.227.2:4127): protocol: rpc: dial tcp 145.79.227.2:4127: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565129266 (564h8m30.917914394s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12gs9LRE3oUp1jupCw5uDvRtkNcahbq2d4VTYQsdKr6atk6vGJx, address:145.79.227.19:4095): protocol: rpc: dial tcp 145.79.227.19:4095: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565148231 (564h8m30.917933359s) [seg_pos={0 0}, upload=496461]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:16:55.580701268 (564h8m30.933486366s) [upload=496461]: Done uploading segments... Sep 22 02:16:55.580714373 (564h8m30.93349947s) [upload=496461]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 437.278833ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [.................................................X.XXXXXX..X] duration: 1m0.017324593s FAIL (2025-09-22 02:16:55 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== m30.954857157s) [seg_pos={0 0}, upload=496461]: Begin upload segment... Sep 22 02:15:55.602116061 (564h7m30.954901189s) [seg_pos={0 0}, upload=496461]: Done begin upload segment. Sep 22 02:15:55.602137471 (564h7m30.954922599s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602154613 (564h7m30.954939742s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602716609 (564h7m30.955501736s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.602736295 (564h7m30.955521423s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603134873 (564h7m30.955920001s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.603640462 (564h7m30.95642559s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604036745 (564h7m30.956821874s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604397323 (564h7m30.95718245s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.604828131 (564h7m30.957613268s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605250674 (564h7m30.958035802s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.605663548 (564h7m30.958448676s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606022221 (564h7m30.958807349s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606387297 (564h7m30.959172425s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.606869060 (564h7m30.959654198s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607319626 (564h7m30.960104755s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.607717662 (564h7m30.960502801s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608211139 (564h7m30.960996268s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.608652857 (564h7m30.961437986s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Uploading piece... Sep 22 02:15:55.704831022 (564h7m31.05761613s) [upload=496461]: Next returned nil segment Sep 22 02:15:55.704846542 (564h7m31.05763165s) [upload=496461]: Waiting for error group managing segments... Sep 22 02:15:55.712709156 (564h7m31.065494285s) [noise=true , address=145.79.227.24:4585 , piece_id=22SO44JH35XXWTOWZE5ZFWGOJ3BSOCUQU4FBPVNQXOMSUKMIYCUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.713081926 (564h7m31.065867054s) [noise=true , address=145.79.227.49:4340 , piece_id=IWZX454IBRQDZX2JXYMSA22AQ43QMZXTWK5UPFEW4V3N3N7GRUGA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.716351481 (564h7m31.06913661s) [noise=true , address=145.79.227.51:4421 , piece_id=HLQZRSFQIG7MP2Q2IK5FYFPCHSSRZAV7HKQX76OFMZBODRUZGZLA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717069920 (564h7m31.069855048s) [noise=true , address=77.90.31.19:4848 , piece_id=RVWFPTS4Q665OF7M3BOWO3AT7HRUDWN6GLDWVSRFJVZHXVA6GI5A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.717303427 (564h7m31.070088556s) [noise=true , address=77.90.31.11:4566 , piece_id=HNEAKRRKHSEZPZIBBSAP2CWIZK572QXGNDON7HWIFFY7227FWILA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725190529 (564h7m31.077975657s) [noise=true , address=145.79.227.39:4019 , piece_id=GT6GJEGLQXJFPHDRBLXUOEBLBB557IZJQQTIF7PWNSXTDORY3K2Q, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.725273014 (564h7m31.078058142s) [noise=true , address=145.79.227.46:4622 , piece_id=2NPEFNHQCXKLTSVFECS7V5WB37KD43JDP5XG3WCLBN4USPPX3NZA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732220611 (564h7m31.085005749s) [noise=true , address=145.79.227.48:4333 , piece_id=ZK4ZZWYUP4BBFZ27YEOTAZ3TBBOFOD2AG6YJDYGZKKDG3CFVUEUA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.732337590 (564h7m31.085122719s) [noise=true , address=145.79.227.33:4901 , piece_id=HKQVMOTQYVYMEAKGSWA4H6QCQX3ND4HC4HNWU2H3UQP2KDESMLTA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.734847580 (564h7m31.087632718s) [noise=true , address=145.79.227.23:4561 , piece_id=GLOGFOAPL3S2J52MIWATNSLAJMKURD4LNTUMEBI3UTGWLZMXU6RQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:15:55.738226892 (564h7m31.09101203s) [noise=true , address=145.79.227.38:4989 , piece_id=VDT3IWTOZEWRJ5PVM5ZEIBYXA6GJ4VMDPWCXWCSZEKF63TV6TC2A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: <nil> Sep 22 02:16:55.564886089 (564h8m30.917671218s) [noise=true , address=145.79.227.12:4790 , piece_id=UR6MYMBXBUAKFOJ2DLM276I37C7Q6WL2GPEOPWJEKOGNWGGYVEMA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1AowWwVvxzmnW5qwbqbMBVbvNoigffjAN51LQp8e6XA9CZkeXL, address:145.79.227.12:4790): protocol: rpc: dial tcp 145.79.227.12:4790: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564944549 (564h8m30.917729677s) [noise=true , address=145.79.227.12:4769 , piece_id=HBMWIEKRPMRTRUFADPYGO6N7MAS2WD4OQMSIDIM7GLBN7QHI2KWA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.564994583 (564h8m30.917779711s) [noise=true , address=145.79.227.15:4896 , piece_id=IMRIZ6K2I7EOQVVYB2IS5IUNCPVYNIMEYUEV5HEJ6GLR7CD5RIHA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565058042 (564h8m30.91784317s) [noise=true , address=145.79.227.8:4349 , piece_id=22ZIRI6DICL7VIVMR2ASWUYA636BB6HUOBGTKLFPKWVCXVB3WUQA, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:1zywsXJnw5gfpUu7mC8p7UE7ev5HexPu5nSpUH1x2pzar99T97, address:145.79.227.8:4349): protocol: rpc: dial tcp 145.79.227.8:4349: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565089361 (564h8m30.917874489s) [noise=true , address=145.79.227.14:4847 , piece_id=3ODTMU7M6VN3UJ6MN2MVAQO64BFVHMACY7FSEXFPOEJXQ7DADKNQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12aAq1Axcts5JgFR2ve4AmPACyCprGsyzM1PERjTYeNM4xeBH1i, address:145.79.227.14:4847): protocol: rpc: dial tcp 145.79.227.14:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565111382 (564h8m30.91789651s) [noise=true , address=145.79.227.2:4127 , piece_id=MSJ6AR3RALXWWBL55L6LBBO26WJBPBFUOBV6HHNWKNLJKB5O2OPQ, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12Pxos2xcjHwcYyQnXeRL3PL72G9pXaAnBs8XnkVFkxegDRPrqp, address:145.79.227.2:4127): protocol: rpc: dial tcp 145.79.227.2:4127: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565129266 (564h8m30.917914394s) [noise=true , address=145.79.227.19:4095 , piece_id=D24Y6HEC4YSP5EZEL764TLAHKAAH6MF22PA5CFFJURCV6VFBTY3A, seg_pos={0 0}, upload=496461]: Done uploading piece. err: ecclient: upload failed (node:12gs9LRE3oUp1jupCw5uDvRtkNcahbq2d4VTYQsdKr6atk6vGJx, address:145.79.227.19:4095): protocol: rpc: dial tcp 145.79.227.19:4095: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:55.565148231 (564h8m30.917933359s) [seg_pos={0 0}, upload=496461]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:16:55.580701268 (564h8m30.933486366s) [upload=496461]: Done uploading segments... Sep 22 02:16:55.580714373 (564h8m30.93349947s) [upload=496461]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 1m0.017344582s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..............................................XXX....XXX.XXX] duration: 30.017711242s FAIL (2025-09-22 02:16:43 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:16:13.330374153 (564h7m48.683159291s) [upload=496466]: Starting upload Sep 22 02:16:13.330489409 (564h7m48.683274537s) [upload=496466]: Done starting upload Sep 22 02:16:13.330587803 (564h7m48.683372932s) [upload=496466]: Uploading segments... Sep 22 02:16:13.330856146 (564h7m48.683641275s) [upload=496466]: Got next segment. Inline: false Sep 22 02:16:13.330888126 (564h7m48.683673255s) [upload=496466]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:16:13.330896101 (564h7m48.68368124s) [upload=496466]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:16:13.358478461 (564h7m48.71126359s) [seg_pos={0 0}, upload=496466]: Begin upload segment... Sep 22 02:16:13.358525650 (564h7m48.711310778s) [seg_pos={0 0}, upload=496466]: Done begin upload segment. Sep 22 02:16:13.358555245 (564h7m48.711340374s) [noise=true , address=45.91.116.12:4226 , piece_id=ZTBWOMPOW6QQT7R6Z3P4YKTNPCDHCFTYAKV6RAY3GRUUUL64CUXA, seg_pos={0 0}, upload=496466]: Uploading piece... Sep 22 02:16:13.358568149 (564h7m48.711353278s) [noise=true , address=77.90.31.18:4805 , piece_id=FJLBQTRKPLV4YYSM62X6KBH57UVJEAYBLMSAXLDTLJESEH522JYQ, seg_pos={0 0}, upload=496466]: Uploading piece... Sep 22 02:16:13.359045766 (564h7m48.711830895s) [noise=true , address=145.79.227.7:4293 , piece_id=MAJRKWFTXMLQ2NJJEOTL4ALCVQR5U5BVAU2NHGIGOBHXQUXQ425A, seg_pos={0 0}, upload=496466]: Uploading piece... Sep 22 02:16:13.359054373 (564h7m48.711839511s) [noise=true , address=45.91.116.9:4261 , piece_id=ENFTXAJHQKGXH63Q7YK3HU5E4C55DTH767K3GGSAJ6ZUFFNY4M7Q, seg_pos={0 0}, upload=496466]: Uploading piece... Sep 22 02:16:13.359420008 (564h7m48.712205147s) [noise=true , address=45.91.116.11:4118 , piece_id=HNGVZGJYNVJLCWOSMJGA3AWPUY2ZUWUVBWV24PEEZQ44MENVQNTA, seg_pos={0 0}, upload=496466]: Uploading piece... Sep 22 02:16:13.359482967 (564h7m48.712268094s) [noise=true , address=145.79.227.17:4972 , piece_id=5ZC3BJC2QF22654HXLR4Y7BU5WLJOBVKPRMMY4KW5FWF2PBSC3RQ, seg_pos={0 0}, upload=496466]: Uploading piece... Sep 22 02:16:13.359800473 (564h7m48.712585601s) [noise=true , address=45.91.116.13:4154 , piece_id=HIGPACRYDDTSUCF62XOGZ3ZVZFRCRTZ2YFJARR7MUCD5PEJSXDNA, seg_pos={0 0}, upload=496466]: Uploading piece... Sep 22 02:16:13.359847882 (564h7m48.71263301s) [noise=true , address=145.79.227.18:4005 , piece_id=EPVH3WEA5UCRLRCO2AWSJBTXQGG3QIPQ3MS6PMAXRNNNFS5JJFVA, seg_pos={0 0}, upload=496466]: Uploading piece... Sep 22 02:16:13.360198019 (564h7m48.712983147s) [noise=true , address=45.91.116.4:4940 , piece_id=LKDVRJY7HZLCAP4GWS6ZDZTNJSSYCBLFEQCBVEHKWNJWJKQFBU6A, seg_pos={0 0}, upload=496466]: Uploading piece... Sep 22 02:16:13.360292075 (564h7m48.713077213s) [noise=true , address=145.79.227.19:4079 , piece_id=RNC46JZCRWMS3VZRIBZEPYTVF5E5PZZYIF3HCTNODCUO474567FA, seg_pos={0 0}, upload=496466]: Uploading piece... Sep 22 02:16:13.360602347 (564h7m48.713387475s) [noise=true , address=45.91.116.10:4081 , piece_id=F7XVZMPHATA7GCTBSBLOV7JRVDSQSJ2CWQ2O6HMYTMGDOIP4K43Q, seg_pos={0 0}, upload=496466]: Uploading piece... Sep 22 02:16:13.360666148 (564h7m48.713451275s) [noise=true , address=45.91.116.5:4299 , piece_id=IBYIKSVE3MIVE5ZSRYKJWPCO5ZWIAZVC6XW6UUHXJCN6MHPHLPVQ, seg_pos={0 0}, upload=496466]: Uploading piece... Sep 22 02:16:13.360958265 (564h7m48.713743394s) [noise=true , address=77.90.31.19:4842 , piece_id=3EXIZTYNOY6MA67N6XPMROUGYCRTCKM4PYK7KFM5VI4SM3N6LNPQ, seg_pos={0 0}, upload=496466]: Uploading piece... Sep 22 02:16:13.361037213 (564h7m48.713822342s) [noise=true , address=45.91.116.8:4047 , piece_id=4D565CITH54WM2BG2LCGFDCE4FOYLJHFKNT6REYI65DNNYMX2V5Q, seg_pos={0 0}, upload=496466]: Uploading piece... Sep 22 02:16:13.361302371 (564h7m48.714087499s) [noise=true , address=45.91.116.14:4190 , piece_id=JZ5CQNGQIHI4JNCZX54OOQXJLSPRE6FBBRKWQV657HRCXDI2QGTA, seg_pos={0 0}, upload=496466]: Uploading piece... Sep 22 02:16:13.361381800 (564h7m48.714166928s) [noise=true , address=77.90.31.21:4878 , piece_id=NTZVHKPKSYVLF6WD2T7C36INVLS5OV6BKIS4ZST4HHV4HNFVDOFA, seg_pos={0 0}, upload=496466]: Uploading piece... Sep 22 02:16:13.470488436 (564h7m48.823273564s) [upload=496466]: Next returned nil segment Sep 22 02:16:13.470504085 (564h7m48.823289214s) [upload=496466]: Waiting for error group managing segments... Sep 22 02:16:13.479535273 (564h7m48.832320412s) [noise=true , address=45.91.116.8:4047 , piece_id=4D565CITH54WM2BG2LCGFDCE4FOYLJHFKNT6REYI65DNNYMX2V5Q, seg_pos={0 0}, upload=496466]: Done uploading piece. err: <nil> Sep 22 02:16:13.480503390 (564h7m48.833288518s) [noise=true , address=77.90.31.21:4878 , piece_id=NTZVHKPKSYVLF6WD2T7C36INVLS5OV6BKIS4ZST4HHV4HNFVDOFA, seg_pos={0 0}, upload=496466]: Done uploading piece. err: <nil> Sep 22 02:16:13.480775571 (564h7m48.833560699s) [noise=true , address=45.91.116.13:4154 , piece_id=HIGPACRYDDTSUCF62XOGZ3ZVZFRCRTZ2YFJARR7MUCD5PEJSXDNA, seg_pos={0 0}, upload=496466]: Done uploading piece. err: <nil> Sep 22 02:16:13.481126800 (564h7m48.833911927s) [noise=true , address=45.91.116.11:4118 , piece_id=HNGVZGJYNVJLCWOSMJGA3AWPUY2ZUWUVBWV24PEEZQ44MENVQNTA, seg_pos={0 0}, upload=496466]: Done uploading piece. err: <nil> Sep 22 02:16:13.481684586 (564h7m48.834469715s) [noise=true , address=45.91.116.12:4226 , piece_id=ZTBWOMPOW6QQT7R6Z3P4YKTNPCDHCFTYAKV6RAY3GRUUUL64CUXA, seg_pos={0 0}, upload=496466]: Done uploading piece. err: <nil> Sep 22 02:16:13.481877298 (564h7m48.834662436s) [noise=true , address=45.91.116.14:4190 , piece_id=JZ5CQNGQIHI4JNCZX54OOQXJLSPRE6FBBRKWQV657HRCXDI2QGTA, seg_pos={0 0}, upload=496466]: Done uploading piece. err: <nil> Sep 22 02:16:13.482208419 (564h7m48.834993547s) [noise=true , address=45.91.116.9:4261 , piece_id=ENFTXAJHQKGXH63Q7YK3HU5E4C55DTH767K3GGSAJ6ZUFFNY4M7Q, seg_pos={0 0}, upload=496466]: Done uploading piece. err: <nil> Sep 22 02:16:13.482693029 (564h7m48.835478157s) [noise=true , address=45.91.116.10:4081 , piece_id=F7XVZMPHATA7GCTBSBLOV7JRVDSQSJ2CWQ2O6HMYTMGDOIP4K43Q, seg_pos={0 0}, upload=496466]: Done uploading piece. err: <nil> Sep 22 02:16:13.482823544 (564h7m48.835608672s) [noise=true , address=45.91.116.5:4299 , piece_id=IBYIKSVE3MIVE5ZSRYKJWPCO5ZWIAZVC6XW6UUHXJCN6MHPHLPVQ, seg_pos={0 0}, upload=496466]: Done uploading piece. err: <nil> Sep 22 02:16:13.483309475 (564h7m48.836094603s) [noise=true , address=45.91.116.4:4940 , piece_id=LKDVRJY7HZLCAP4GWS6ZDZTNJSSYCBLFEQCBVEHKWNJWJKQFBU6A, seg_pos={0 0}, upload=496466]: Done uploading piece. err: <nil> Sep 22 02:16:13.483998728 (564h7m48.836783857s) [noise=true , address=77.90.31.19:4842 , piece_id=3EXIZTYNOY6MA67N6XPMROUGYCRTCKM4PYK7KFM5VI4SM3N6LNPQ, seg_pos={0 0}, upload=496466]: Done uploading piece. err: <nil> Sep 22 02:16:13.485052246 (564h7m48.837837373s) [noise=true , address=77.90.31.18:4805 , piece_id=FJLBQTRKPLV4YYSM62X6KBH57UVJEAYBLMSAXLDTLJESEH522JYQ, seg_pos={0 0}, upload=496466]: Done uploading piece. err: <nil> Sep 22 02:16:43.314692359 (564h8m18.667477487s) [noise=true , address=145.79.227.7:4293 , piece_id=MAJRKWFTXMLQ2NJJEOTL4ALCVQR5U5BVAU2NHGIGOBHXQUXQ425A, seg_pos={0 0}, upload=496466]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:43.314748795 (564h8m18.667533933s) [noise=true , address=145.79.227.19:4079 , piece_id=RNC46JZCRWMS3VZRIBZEPYTVF5E5PZZYIF3HCTNODCUO474567FA, seg_pos={0 0}, upload=496466]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: rpc: dial tcp 145.79.227.19:4079: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:43.314809539 (564h8m18.667594667s) [noise=true , address=145.79.227.17:4972 , piece_id=5ZC3BJC2QF22654HXLR4Y7BU5WLJOBVKPRMMY4KW5FWF2PBSC3RQ, seg_pos={0 0}, upload=496466]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: rpc: dial tcp 145.79.227.17:4972: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:43.314853512 (564h8m18.667638639s) [noise=true , address=145.79.227.18:4005 , piece_id=EPVH3WEA5UCRLRCO2AWSJBTXQGG3QIPQ3MS6PMAXRNNNFS5JJFVA, seg_pos={0 0}, upload=496466]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:43.314889609 (564h8m18.667674737s) [seg_pos={0 0}, upload=496466]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:16:43.329938581 (564h8m18.682723718s) [upload=496466]: Done uploading segments... Sep 22 02:16:43.329975069 (564h8m18.682760198s) [upload=496466]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.01772654s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [...............................................XXX....XXX.XX] duration: 30.022890244s FAIL (2025-09-22 02:16:13 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Sep 22 02:15:43.256038386 (564h7m18.608823515s) [upload=496458]: Starting upload Sep 22 02:15:43.256266474 (564h7m18.609051602s) [upload=496458]: Done starting upload Sep 22 02:15:43.256387372 (564h7m18.6091725s) [upload=496458]: Uploading segments... Sep 22 02:15:43.256660374 (564h7m18.609445502s) [upload=496458]: Got next segment. Inline: false Sep 22 02:15:43.256706911 (564h7m18.609492039s) [upload=496458]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:15:43.256712492 (564h7m18.60949763s) [upload=496458]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:15:43.282570556 (564h7m18.635355684s) [seg_pos={0 0}, upload=496458]: Begin upload segment... Sep 22 02:15:43.282636530 (564h7m18.635421668s) [seg_pos={0 0}, upload=496458]: Done begin upload segment. Sep 22 02:15:43.282666044 (564h7m18.635451173s) [noise=true , address=145.79.227.16:4911 , piece_id=TVASVRP6XD5OZEN4HAZ66IC63GXZW7XSQ37HLJ3VZF4X65VMTEQA, seg_pos={0 0}, upload=496458]: Uploading piece... Sep 22 02:15:43.283006545 (564h7m18.635791672s) [noise=true , address=145.79.227.17:4970 , piece_id=QPFFDYNGLI2QZTAK7TRP3ZGFGL432GZOF4VXEWSVGA7AFC5ZKC5A, seg_pos={0 0}, upload=496458]: Uploading piece... Sep 22 02:15:43.283555654 (564h7m18.636340782s) [noise=true , address=145.79.227.19:4079 , piece_id=CT5HZGIHZHI2CLJ7WA2JYPSPRCJQVOUB2FLRBV74ZOIUUEBEZPCA, seg_pos={0 0}, upload=496458]: Uploading piece... Sep 22 02:15:43.284370884 (564h7m18.637156012s) [noise=true , address=145.79.227.3:4150 , piece_id=OFBQFDBMDC7QVYANNUDXDV2TAGYKNUYBSUV45FD2RGQMIJYDW23Q, seg_pos={0 0}, upload=496458]: Uploading piece... Sep 22 02:15:43.285046432 (564h7m18.63783156s) [noise=true , address=45.91.116.10:4081 , piece_id=RLPRYTUQTLTP52EKFGEZUD4OAZ666GZYF62CGB2FGRVBUEH4J4CQ, seg_pos={0 0}, upload=496458]: Uploading piece... Sep 22 02:15:43.285671756 (564h7m18.638456884s) [noise=true , address=145.79.227.13:4804 , piece_id=4VTIZICOFSVKT7XAU4CHPT6IRU5UGZQUQD6D2HZY2TMRRIXWTOBA, seg_pos={0 0}, upload=496458]: Uploading piece... Sep 22 02:15:43.286381307 (564h7m18.639166435s) [noise=true , address=45.91.116.4:4939 , piece_id=QS6DPU4NF5VBW55TPTPYCPSXNFXCJCLNVZNPVE4YLUK53ZETDSEA, seg_pos={0 0}, upload=496458]: Uploading piece... Sep 22 02:15:43.287055773 (564h7m18.639840911s) [noise=true , address=145.79.227.15:4876 , piece_id=D42BAXQ2GDSUXXSEV2WBJ5N4XBQ7PAU2BNZELOXS6GSTJ242ZCRA, seg_pos={0 0}, upload=496458]: Uploading piece... Sep 22 02:15:43.287784561 (564h7m18.640569689s) [noise=true , address=77.90.31.13:4628 , piece_id=YAEMQXBJWOH6G4RRQWSNG7HUO7WDTC2JEET5XPE54WLJ2EG5WFEQ, seg_pos={0 0}, upload=496458]: Uploading piece... Sep 22 02:15:43.288990152 (564h7m18.641775281s) [noise=true , address=77.90.31.12:4590 , piece_id=VWMSYZAQ7SOYBL6GOTIIKOZ7GMSHVLQSX7T47RL3BBUTITPLTDIQ, seg_pos={0 0}, upload=496458]: Uploading piece... Sep 22 02:15:43.289700085 (564h7m18.642485213s) [noise=true , address=45.91.116.5:4298 , piece_id=LZ5CGD6LTJNZDXJGC2EHBJ4CCPTG64IRSMCGWBBBZ7333LLXKONA, seg_pos={0 0}, upload=496458]: Uploading piece... Sep 22 02:15:43.291797591 (564h7m18.644582719s) [noise=true , address=45.91.116.11:4117 , piece_id=F5KERCSFDLHU2YOONGL4TNY74Y4EPVIETPZLJWLY6BEWSK2BQNRQ, seg_pos={0 0}, upload=496458]: Uploading piece... Sep 22 02:15:43.294901084 (564h7m18.647686213s) [noise=true , address=145.79.227.7:4293 , piece_id=KTSNVROVTJMX7WRBTK6MHSVDEB66XA2X77LXT3GLBJK7YHFXMTYA, seg_pos={0 0}, upload=496458]: Uploading piece... Sep 22 02:15:43.297338890 (564h7m18.650124028s) [noise=true , address=77.90.31.21:4878 , piece_id=AG4CCTLVHBID45IA54OIIH7TF7KBJVYBM5TA3E6BZ7UBVEEBHDFA, seg_pos={0 0}, upload=496458]: Uploading piece... Sep 22 02:15:43.299576248 (564h7m18.652361376s) [noise=true , address=45.91.116.9:4262 , piece_id=JFWVHQIHBIUFLF672A5GNJQKSIB7YYIUT3OQ5ZI2QPU26L2YBXCA, seg_pos={0 0}, upload=496458]: Uploading piece... Sep 22 02:15:43.301951786 (564h7m18.654736915s) [noise=true , address=45.91.116.14:4189 , piece_id=KM2HMD5GCIF53YTI7COQ56Y25G4J6RIQOVYDBTO27OUM4L54BVNQ, seg_pos={0 0}, upload=496458]: Uploading piece... Sep 22 02:15:43.397529484 (564h7m18.750314622s) [upload=496458]: Next returned nil segment Sep 22 02:15:43.397554681 (564h7m18.75033981s) [upload=496458]: Waiting for error group managing segments... Sep 22 02:15:43.407757608 (564h7m18.760542736s) [noise=true , address=45.91.116.10:4081 , piece_id=RLPRYTUQTLTP52EKFGEZUD4OAZ666GZYF62CGB2FGRVBUEH4J4CQ, seg_pos={0 0}, upload=496458]: Done uploading piece. err: <nil> Sep 22 02:15:43.407822360 (564h7m18.760607487s) [noise=true , address=45.91.116.4:4939 , piece_id=QS6DPU4NF5VBW55TPTPYCPSXNFXCJCLNVZNPVE4YLUK53ZETDSEA, seg_pos={0 0}, upload=496458]: Done uploading piece. err: <nil> Sep 22 02:15:43.408488730 (564h7m18.761273857s) [noise=true , address=45.91.116.9:4262 , piece_id=JFWVHQIHBIUFLF672A5GNJQKSIB7YYIUT3OQ5ZI2QPU26L2YBXCA, seg_pos={0 0}, upload=496458]: Done uploading piece. err: <nil> Sep 22 02:15:43.408775738 (564h7m18.761560886s) [noise=true , address=45.91.116.14:4189 , piece_id=KM2HMD5GCIF53YTI7COQ56Y25G4J6RIQOVYDBTO27OUM4L54BVNQ, seg_pos={0 0}, upload=496458]: Done uploading piece. err: <nil> Sep 22 02:15:43.409129372 (564h7m18.7619145s) [noise=true , address=45.91.116.5:4298 , piece_id=LZ5CGD6LTJNZDXJGC2EHBJ4CCPTG64IRSMCGWBBBZ7333LLXKONA, seg_pos={0 0}, upload=496458]: Done uploading piece. err: <nil> Sep 22 02:15:43.409518232 (564h7m18.762303371s) [noise=true , address=45.91.116.11:4117 , piece_id=F5KERCSFDLHU2YOONGL4TNY74Y4EPVIETPZLJWLY6BEWSK2BQNRQ, seg_pos={0 0}, upload=496458]: Done uploading piece. err: <nil> Sep 22 02:15:43.409809959 (564h7m18.762595087s) [noise=true , address=77.90.31.13:4628 , piece_id=YAEMQXBJWOH6G4RRQWSNG7HUO7WDTC2JEET5XPE54WLJ2EG5WFEQ, seg_pos={0 0}, upload=496458]: Done uploading piece. err: <nil> Sep 22 02:15:43.410083512 (564h7m18.762868641s) [noise=true , address=77.90.31.21:4878 , piece_id=AG4CCTLVHBID45IA54OIIH7TF7KBJVYBM5TA3E6BZ7UBVEEBHDFA, seg_pos={0 0}, upload=496458]: Done uploading piece. err: <nil> Sep 22 02:15:43.410505545 (564h7m18.763290683s) [noise=true , address=77.90.31.12:4590 , piece_id=VWMSYZAQ7SOYBL6GOTIIKOZ7GMSHVLQSX7T47RL3BBUTITPLTDIQ, seg_pos={0 0}, upload=496458]: Done uploading piece. err: <nil> Sep 22 02:16:13.239515750 (564h7m48.592300888s) [noise=true , address=145.79.227.19:4079 , piece_id=CT5HZGIHZHI2CLJ7WA2JYPSPRCJQVOUB2FLRBV74ZOIUUEBEZPCA, seg_pos={0 0}, upload=496458]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: rpc: dial tcp 145.79.227.19:4079: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:13.239842212 (564h7m48.59262734s) [noise=true , address=145.79.227.3:4150 , piece_id=OFBQFDBMDC7QVYANNUDXDV2TAGYKNUYBSUV45FD2RGQMIJYDW23Q, seg_pos={0 0}, upload=496458]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: rpc: dial tcp 145.79.227.3:4150: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:13.239891745 (564h7m48.592676873s) [noise=true , address=145.79.227.15:4876 , piece_id=D42BAXQ2GDSUXXSEV2WBJ5N4XBQ7PAU2BNZELOXS6GSTJ242ZCRA, seg_pos={0 0}, upload=496458]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: rpc: dial tcp 145.79.227.15:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:13.239903828 (564h7m48.592688966s) [noise=true , address=145.79.227.17:4970 , piece_id=QPFFDYNGLI2QZTAK7TRP3ZGFGL432GZOF4VXEWSVGA7AFC5ZKC5A, seg_pos={0 0}, upload=496458]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:13.239997415 (564h7m48.592782542s) [noise=true , address=145.79.227.16:4911 , piece_id=TVASVRP6XD5OZEN4HAZ66IC63GXZW7XSQ37HLJ3VZF4X65VMTEQA, seg_pos={0 0}, upload=496458]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: rpc: dial tcp 145.79.227.16:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:13.240030196 (564h7m48.592815324s) [noise=true , address=145.79.227.7:4293 , piece_id=KTSNVROVTJMX7WRBTK6MHSVDEB66XA2X77LXT3GLBJK7YHFXMTYA, seg_pos={0 0}, upload=496458]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:13.240055984 (564h7m48.592841113s) [noise=true , address=145.79.227.13:4804 , piece_id=4VTIZICOFSVKT7XAU4CHPT6IRU5UGZQUQD6D2HZY2TMRRIXWTOBA, seg_pos={0 0}, upload=496458]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:13.240086391 (564h7m48.592871519s) [seg_pos={0 0}, upload=496458]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 22 02:16:13.260554792 (564h7m48.61333993s) [upload=496458]: Done uploading segments... Sep 22 02:16:13.260572195 (564h7m48.613357322s) [upload=496458]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.022912437s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................................X.X.........] duration: 598.250639ms FAIL (2025-09-22 02:16:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 598.270755ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [................................................XXX....XXX.X] duration: 30.018460617s FAIL (2025-09-22 02:15:41 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Sep 22 02:15:11.465871737 (564h6m46.818656864s) [upload=496448]: Starting upload Sep 22 02:15:11.465950313 (564h6m46.818735442s) [upload=496448]: Done starting upload Sep 22 02:15:11.466018512 (564h6m46.818803651s) [upload=496448]: Uploading segments... Sep 22 02:15:11.466339946 (564h6m46.819125073s) [upload=496448]: Got next segment. Inline: false Sep 22 02:15:11.466411079 (564h6m46.819196247s) [upload=496448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:15:11.466419785 (564h6m46.819204914s) [upload=496448]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:15:11.488738338 (564h6m46.841523466s) [seg_pos={0 0}, upload=496448]: Begin upload segment... Sep 22 02:15:11.488774486 (564h6m46.841559624s) [seg_pos={0 0}, upload=496448]: Done begin upload segment. Sep 22 02:15:11.488810373 (564h6m46.841595501s) [noise=true , address=145.79.227.15:4874 , piece_id=IXYVJOEM6ANWUMVVJRZY5SYYL2JCEP6VCXTEBCWRFEML565IATNQ, seg_pos={0 0}, upload=496448]: Uploading piece... Sep 22 02:15:11.488864344 (564h6m46.841649473s) [noise=true , address=145.79.227.11:4730 , piece_id=HGFA5NP7JE4PKW2AFIP4JVZFUUHFRQQEJFDT7IE2TG736IGZTRAA, seg_pos={0 0}, upload=496448]: Uploading piece... Sep 22 02:15:11.489252071 (564h6m46.8420372s) [noise=true , address=77.90.31.12:4589 , piece_id=CN6VVBLRS7TMNG4Q7RG56QRPIUNJYXTTWZMDSVPMEACYS5XHGWJQ, seg_pos={0 0}, upload=496448]: Uploading piece... Sep 22 02:15:11.489658785 (564h6m46.842443924s) [noise=true , address=77.90.31.21:4877 , piece_id=UNOKXT6PVZNDAG7DUOQRITWLQSPRKGJGSTPJAEFEJVVZOFRJFDDA, seg_pos={0 0}, upload=496448]: Uploading piece... Sep 22 02:15:11.489680476 (564h6m46.842465604s) [noise=true , address=145.79.227.18:4005 , piece_id=SB77VPHFTGHHJRIC2BJY4BKNX5YQ4QCZLJC2M4JUHJJSTN3NZ5HA, seg_pos={0 0}, upload=496448]: Uploading piece... Sep 22 02:15:11.490019933 (564h6m46.84280506s) [noise=true , address=45.91.116.8:4048 , piece_id=2AQSKXLDB2PQJHAFDZDOQ743ZL5SVPQRUAO525ODHWFDHK7Q2PNQ, seg_pos={0 0}, upload=496448]: Uploading piece... Sep 22 02:15:11.490277898 (564h6m46.843063025s) [noise=true , address=45.91.116.12:4226 , piece_id=KYP6DB2VATKE2IY2QMKJZ4TS3UMGHMEUPBTCREJRTOFHCKFK56RA, seg_pos={0 0}, upload=496448]: Uploading piece... Sep 22 02:15:11.490370290 (564h6m46.843155429s) [noise=true , address=145.79.227.18:4006 , piece_id=PIYOTG4XEBD5OZAWGGSWQ6XT63JOERDSHHSZF6REIUVOZWMEIF5A, seg_pos={0 0}, upload=496448]: Uploading piece... Sep 22 02:15:11.490747518 (564h6m46.843532646s) [noise=true , address=45.91.116.14:4189 , piece_id=6RI5YBU46E4GYNLJP55H7MUDFTDSAULLOBHBRSR2PYPTHGTE3YUQ, seg_pos={0 0}, upload=496448]: Uploading piece... Sep 22 02:15:11.490837757 (564h6m46.843622885s) [noise=true , address=45.91.116.6:4973 , piece_id=EKVHHZWL7Q4RZ4XJ5VWNPSZAWTVOJHHNZAFDCH7XKBDKUOUS65VA, seg_pos={0 0}, upload=496448]: Uploading piece... Sep 22 02:15:11.491196371 (564h6m46.843981499s) [noise=true , address=77.90.31.19:4842 , piece_id=IV6WHIXBLCTHPCSPQIQW4ZKPFDH7HSUBZKE5LSTZKENEWOEPM5EA, seg_pos={0 0}, upload=496448]: Uploading piece... Sep 22 02:15:11.491389994 (564h6m46.844175132s) [noise=true , address=77.90.31.10:4519 , piece_id=DWA4UTOHCUEOHMCG4BYG4AQZN3GJHS6NY36CRGP322DRQRHV72DA, seg_pos={0 0}, upload=496448]: Uploading piece... Sep 22 02:15:11.491585691 (564h6m46.844370829s) [noise=true , address=145.79.227.4:4186 , piece_id=XVE4ORGB7OA3AM6B4DGGUETWH537AV5JDOSXNZ4VTIWJRHGBQZMQ, seg_pos={0 0}, upload=496448]: Uploading piece... Sep 22 02:15:11.491987565 (564h6m46.844772693s) [noise=true , address=45.91.116.14:4191 , piece_id=KLPJR3LWMAEX2ISCDGYREYL262KEBPVMJFPWVPIDK26SIXNOXYYQ, seg_pos={0 0}, upload=496448]: Uploading piece... Sep 22 02:15:11.492042698 (564h6m46.844827826s) [noise=true , address=145.79.227.3:4149 , piece_id=RTPCY4MJR7IU447YGHUAGI7ZRRHZO36JOAPDBGNHZD7D3KOHBZEA, seg_pos={0 0}, upload=496448]: Uploading piece... Sep 22 02:15:11.492600115 (564h6m46.845385243s) [noise=true , address=145.79.227.15:4873 , piece_id=G2MU4DWP4IKJA262QYRM77WJASUK2HNCSJ5TK3DK7LZ6WZLS6C3A, seg_pos={0 0}, upload=496448]: Uploading piece... Sep 22 02:15:11.600661939 (564h6m46.953447068s) [upload=496448]: Next returned nil segment Sep 22 02:15:11.600677218 (564h6m46.953462346s) [upload=496448]: Waiting for error group managing segments... Sep 22 02:15:11.609549919 (564h6m46.962335057s) [noise=true , address=45.91.116.12:4226 , piece_id=KYP6DB2VATKE2IY2QMKJZ4TS3UMGHMEUPBTCREJRTOFHCKFK56RA, seg_pos={0 0}, upload=496448]: Done uploading piece. err: <nil> Sep 22 02:15:11.611032621 (564h6m46.963817749s) [noise=true , address=45.91.116.6:4973 , piece_id=EKVHHZWL7Q4RZ4XJ5VWNPSZAWTVOJHHNZAFDCH7XKBDKUOUS65VA, seg_pos={0 0}, upload=496448]: Done uploading piece. err: <nil> Sep 22 02:15:11.613257717 (564h6m46.966042845s) [noise=true , address=77.90.31.19:4842 , piece_id=IV6WHIXBLCTHPCSPQIQW4ZKPFDH7HSUBZKE5LSTZKENEWOEPM5EA, seg_pos={0 0}, upload=496448]: Done uploading piece. err: <nil> Sep 22 02:15:11.613567928 (564h6m46.966353057s) [noise=true , address=45.91.116.14:4191 , piece_id=KLPJR3LWMAEX2ISCDGYREYL262KEBPVMJFPWVPIDK26SIXNOXYYQ, seg_pos={0 0}, upload=496448]: Done uploading piece. err: <nil> Sep 22 02:15:11.614658294 (564h6m46.967443423s) [noise=true , address=45.91.116.14:4189 , piece_id=6RI5YBU46E4GYNLJP55H7MUDFTDSAULLOBHBRSR2PYPTHGTE3YUQ, seg_pos={0 0}, upload=496448]: Done uploading piece. err: <nil> Sep 22 02:15:11.615321660 (564h6m46.968106788s) [noise=true , address=77.90.31.21:4877 , piece_id=UNOKXT6PVZNDAG7DUOQRITWLQSPRKGJGSTPJAEFEJVVZOFRJFDDA, seg_pos={0 0}, upload=496448]: Done uploading piece. err: <nil> Sep 22 02:15:11.615684490 (564h6m46.968469619s) [noise=true , address=77.90.31.12:4589 , piece_id=CN6VVBLRS7TMNG4Q7RG56QRPIUNJYXTTWZMDSVPMEACYS5XHGWJQ, seg_pos={0 0}, upload=496448]: Done uploading piece. err: <nil> Sep 22 02:15:11.616327858 (564h6m46.969112986s) [noise=true , address=77.90.31.10:4519 , piece_id=DWA4UTOHCUEOHMCG4BYG4AQZN3GJHS6NY36CRGP322DRQRHV72DA, seg_pos={0 0}, upload=496448]: Done uploading piece. err: <nil> Sep 22 02:15:11.617010228 (564h6m46.969795357s) [noise=true , address=45.91.116.8:4048 , piece_id=2AQSKXLDB2PQJHAFDZDOQ743ZL5SVPQRUAO525ODHWFDHK7Q2PNQ, seg_pos={0 0}, upload=496448]: Done uploading piece. err: <nil> Sep 22 02:15:41.451046320 (564h7m16.803831448s) [noise=true , address=145.79.227.15:4874 , piece_id=IXYVJOEM6ANWUMVVJRZY5SYYL2JCEP6VCXTEBCWRFEML565IATNQ, seg_pos={0 0}, upload=496448]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: rpc: dial tcp 145.79.227.15:4874: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:15:41.451110721 (564h7m16.803895849s) [noise=true , address=145.79.227.11:4730 , piece_id=HGFA5NP7JE4PKW2AFIP4JVZFUUHFRQQEJFDT7IE2TG736IGZTRAA, seg_pos={0 0}, upload=496448]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:15:41.451167156 (564h7m16.803952285s) [noise=true , address=145.79.227.18:4006 , piece_id=PIYOTG4XEBD5OZAWGGSWQ6XT63JOERDSHHSZF6REIUVOZWMEIF5A, seg_pos={0 0}, upload=496448]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: rpc: dial tcp 145.79.227.18:4006: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:15:41.451231488 (564h7m16.804016615s) [noise=true , address=145.79.227.4:4186 , piece_id=XVE4ORGB7OA3AM6B4DGGUETWH537AV5JDOSXNZ4VTIWJRHGBQZMQ, seg_pos={0 0}, upload=496448]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:15:41.451277704 (564h7m16.804062832s) [noise=true , address=145.79.227.3:4149 , piece_id=RTPCY4MJR7IU447YGHUAGI7ZRRHZO36JOAPDBGNHZD7D3KOHBZEA, seg_pos={0 0}, upload=496448]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: rpc: dial tcp 145.79.227.3:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:15:41.451312499 (564h7m16.804097628s) [noise=true , address=145.79.227.15:4873 , piece_id=G2MU4DWP4IKJA262QYRM77WJASUK2HNCSJ5TK3DK7LZ6WZLS6C3A, seg_pos={0 0}, upload=496448]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:15:41.451346954 (564h7m16.804132082s) [noise=true , address=145.79.227.18:4005 , piece_id=SB77VPHFTGHHJRIC2BJY4BKNX5YQ4QCZLJC2M4JUHJJSTN3NZ5HA, seg_pos={0 0}, upload=496448]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 02:15:41.451375137 (564h7m16.804160265s) [seg_pos={0 0}, upload=496448]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 22 02:15:41.467635059 (564h7m16.820420188s) [upload=496448]: Done uploading segments... Sep 22 02:15:41.467653454 (564h7m16.820438582s) [upload=496448]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.018486624s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................................X.X........] duration: 451.839488ms FAIL (2025-09-22 02:15:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 451.856921ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 7 out of 60 is failed (threshold=3) [..................................................X.XXXXXX..] duration: 860.187749ms FAIL (2025-09-22 02:15:22 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Sep 22 02:13:31.123370419 (564h5m6.476155547s) [upload=496426]: Starting upload Sep 22 02:13:31.123482970 (564h5m6.476268108s) [upload=496426]: Done starting upload Sep 22 02:13:31.123900823 (564h5m6.476685952s) [upload=496426]: Uploading segments... Sep 22 02:13:31.123976205 (564h5m6.476761333s) [upload=496426]: Got next segment. Inline: false Sep 22 02:13:31.124008105 (564h5m6.476793233s) [upload=496426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:13:31.124015108 (564h5m6.476800236s) [upload=496426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:13:31.148379950 (564h5m6.501165049s) [seg_pos={0 0}, upload=496426]: Begin upload segment... Sep 22 02:13:31.148424965 (564h5m6.501210063s) [seg_pos={0 0}, upload=496426]: Done begin upload segment. Sep 22 02:13:31.148467605 (564h5m6.501252703s) [noise=true , address=145.79.227.28:4729 , piece_id=3BFJAEVK4MDGJPLI5FQX7GTFVP7G5VE5EPVI4CKBG6TXNDRJPO6A, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.148650889 (564h5m6.501436016s) [noise=true , address=145.79.227.25:4651 , piece_id=XBVPDQH5NOTHMMQPBAHF5MIJTXVGU37LLJTF35HEXYEN6AEIMV3A, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.148844502 (564h5m6.50162961s) [noise=true , address=145.79.227.17:4997 , piece_id=BDNGRBTCNQK7AOFEHP4IUDTHLH2HPN2P6XYL25NPAYGAFKDIHZ6Q, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.149209627 (564h5m6.501994736s) [noise=true , address=145.79.227.30:4764 , piece_id=IW5TJOGT3AAKO2PBZXVW6PN3FUS4LM23VGVPYYUNNIQUE3XUBMXQ, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.149249272 (564h5m6.5020344s) [noise=true , address=145.79.227.46:4602 , piece_id=XNKJNNJASPEHCR3UVDTU55QFKVPNRWLDS3E3POQ7LUG5SX6IKUZA, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.149564203 (564h5m6.502349301s) [noise=true , address=145.79.227.12:4785 , piece_id=3LQDV6LTJBIKUMSHHJWOUCNVV2ZCLXBEYCVL4XFMUIN7374AIJBQ, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.149758056 (564h5m6.502543185s) [noise=true , address=145.79.227.45:4258 , piece_id=HJCI2PBCN27XBTXUJPYV7MADQKKKX5CB3TVAKXITEH3CCA7TZSVA, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.149834681 (564h5m6.502619779s) [noise=true , address=145.79.227.5:4236 , piece_id=KVODBMFFVHK2PUHPGVLREOV5SK7YE7B2IYXVECYDI655GUWBEUJQ, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.150103565 (564h5m6.502888674s) [noise=true , address=145.79.227.16:4915 , piece_id=77ESS2CVFF62METXPWC5NNB54L6MOPCGFTNYBY7AHJQRPW76CYLA, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.150228108 (564h5m6.503013237s) [noise=true , address=145.79.227.27:4716 , piece_id=QU2NFV24XNCXFCQAKKX32TRAVEGKBWCAZXQ2OZFNAEMV7M544I2A, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.150355278 (564h5m6.503140386s) [noise=true , address=145.79.227.35:4512 , piece_id=DZ3UZX2WCHGG2AIAI6QU6GDET42XYN3OQ72VT654ULTNHQ3WXZ7A, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.150621356 (564h5m6.503406464s) [noise=true , address=145.79.227.23:4556 , piece_id=4OJ2XWKI6RFRWKKTEK6AD2MHVHWFCFJ6GSM6P4TBJ3APXL4C6HEQ, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.150786868 (564h5m6.503571996s) [noise=true , address=145.79.227.26:4653 , piece_id=FLZGOCSUBA4DYJWANSNQFKAYD723ETVVQVZ65DBAYGZ6MEMU7BQQ, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.150893007 (564h5m6.503678104s) [noise=true , address=145.79.227.49:4347 , piece_id=RJ3WB565YG3WCEPEAS7JT557G3UKSFS3ZVOT7GIV2222PDVFYTPQ, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.151216654 (564h5m6.504001782s) [noise=true , address=145.79.227.16:4917 , piece_id=IJMWS6SWG45VPUAISPJVHKPA6FUS4QQMR5D3WCHW54XDHSEPM24A, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.163881771 (564h5m6.516666879s) [noise=true , address=145.79.227.52:4465 , piece_id=SVLLR6OMTR75SPEPG5ULYYHQRP5AFJHDOIF7ME5ZGLCMS6ZES43A, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.164384484 (564h5m6.517169592s) [noise=true , address=145.79.227.43:4167 , piece_id=WLBLSW6LWMGHKU5LWYLQLDRFOZXTIEHWUJDZMMZVEWATWUO5M32A, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.164728540 (564h5m6.517513639s) [noise=true , address=145.79.227.48:4336 , piece_id=EZZILAKKPJOIOLV23GBINEHTN752RNOC4WFS3PUSYNVW2IYVDQ3Q, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.221725048 (564h5m6.574510146s) [upload=496426]: Next returned nil segment Sep 22 02:13:31.221737140 (564h5m6.574522238s) [upload=496426]: Waiting for error group managing segments... Sep 22 02:13:31.236128126 (564h5m6.588913224s) [noise=true , address=145.79.227.27:4716 , piece_id=QU2NFV24XNCXFCQAKKX32TRAVEGKBWCAZXQ2OZFNAEMV7M544I2A, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.240486955 (564h5m6.593272073s) [noise=true , address=145.79.227.28:4729 , piece_id=3BFJAEVK4MDGJPLI5FQX7GTFVP7G5VE5EPVI4CKBG6TXNDRJPO6A, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.241190977 (564h5m6.593976074s) [noise=true , address=145.79.227.30:4764 , piece_id=IW5TJOGT3AAKO2PBZXVW6PN3FUS4LM23VGVPYYUNNIQUE3XUBMXQ, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.250713206 (564h5m6.603498324s) [noise=true , address=145.79.227.46:4602 , piece_id=XNKJNNJASPEHCR3UVDTU55QFKVPNRWLDS3E3POQ7LUG5SX6IKUZA, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.251333439 (564h5m6.604118558s) [noise=true , address=145.79.227.26:4653 , piece_id=FLZGOCSUBA4DYJWANSNQFKAYD723ETVVQVZ65DBAYGZ6MEMU7BQQ, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.251939377 (564h5m6.604724475s) [noise=true , address=145.79.227.23:4556 , piece_id=4OJ2XWKI6RFRWKKTEK6AD2MHVHWFCFJ6GSM6P4TBJ3APXL4C6HEQ, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.253130262 (564h5m6.605915359s) [noise=true , address=145.79.227.35:4512 , piece_id=DZ3UZX2WCHGG2AIAI6QU6GDET42XYN3OQ72VT654ULTNHQ3WXZ7A, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.258561514 (564h5m6.611346612s) [noise=true , address=145.79.227.45:4258 , piece_id=HJCI2PBCN27XBTXUJPYV7MADQKKKX5CB3TVAKXITEH3CCA7TZSVA, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.263589358 (564h5m6.616374487s) [noise=true , address=145.79.227.52:4465 , piece_id=SVLLR6OMTR75SPEPG5ULYYHQRP5AFJHDOIF7ME5ZGLCMS6ZES43A, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.267382377 (564h5m6.620167475s) [noise=true , address=145.79.227.25:4651 , piece_id=XBVPDQH5NOTHMMQPBAHF5MIJTXVGU37LLJTF35HEXYEN6AEIMV3A, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.270822481 (564h5m6.62360761s) [noise=true , address=145.79.227.48:4336 , piece_id=EZZILAKKPJOIOLV23GBINEHTN752RNOC4WFS3PUSYNVW2IYVDQ3Q, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.271707231 (564h5m6.62449236s) [noise=true , address=145.79.227.49:4347 , piece_id=RJ3WB565YG3WCEPEAS7JT557G3UKSFS3ZVOT7GIV2222PDVFYTPQ, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.272633100 (564h5m6.625418227s) [noise=true , address=145.79.227.43:4167 , piece_id=WLBLSW6LWMGHKU5LWYLQLDRFOZXTIEHWUJDZMMZVEWATWUO5M32A, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:14:31.101811916 (564h6m6.454597014s) [noise=true , address=145.79.227.16:4917 , piece_id=IJMWS6SWG45VPUAISPJVHKPA6FUS4QQMR5D3WCHW54XDHSEPM24A, seg_pos={0 0}, upload=496426]: Done uploading piece. err: ecclient: upload failed (node:1VaqXNoBYjwbbE9xhAoLzDuxZMdBPSm2aMw7bYm4qZqiEGp5CV, address:145.79.227.16:4917): protocol: rpc: dial tcp 145.79.227.16:4917: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:14:31.101898368 (564h6m6.454683476s) [noise=true , address=145.79.227.5:4236 , piece_id=KVODBMFFVHK2PUHPGVLREOV5SK7YE7B2IYXVECYDI655GUWBEUJQ, seg_pos={0 0}, upload=496426]: Done uploading piece. err: ecclient: upload failed (node:1XaE29W27ym2RmuGdBvHeKcQZZEUbask1zsm9Bz8xLokZNGfcj, address:145.79.227.5:4236): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:14:31.101937822 (564h6m6.454722919s) [noise=true , address=145.79.227.16:4915 , piece_id=77ESS2CVFF62METXPWC5NNB54L6MOPCGFTNYBY7AHJQRPW76CYLA, seg_pos={0 0}, upload=496426]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: rpc: dial tcp 145.79.227.16:4915: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:14:31.101956186 (564h6m6.454741294s) [noise=true , address=145.79.227.17:4997 , piece_id=BDNGRBTCNQK7AOFEHP4IUDTHLH2HPN2P6XYL25NPAYGAFKDIHZ6Q, seg_pos={0 0}, upload=496426]: Done uploading piece. err: ecclient: upload failed (node:12UrPGB76X7LaXe3ioiu4NtUQ5FwspKYtBWD6rcMmZC6xbNnCAF, address:145.79.227.17:4997): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:14:31.101961587 (564h6m6.454746695s) [noise=true , address=145.79.227.12:4785 , piece_id=3LQDV6LTJBIKUMSHHJWOUCNVV2ZCLXBEYCVL4XFMUIN7374AIJBQ, seg_pos={0 0}, upload=496426]: Done uploading piece. err: ecclient: upload failed (node:12u5zY7F7oy2tPUyFfoiebzFEySetBks9LC3gRGPURAyDDnsUu9, address:145.79.227.12:4785): protocol: rpc: dial tcp 145.79.227.12:4785: i/o timeout; context deadline exceeded Sep 22 02:14:31.101981153 (564h6m6.454766261s) [seg_pos={0 0}, upload=496426]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 02:14:31.118073471 (564h6m6.47085857s) [upload=496426]: Done uploading segments... Sep 22 02:14:31.118083610 (564h6m6.470868718s) [upload=496426]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 860.216133ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 7 out of 60 is failed (threshold=3) [...................................................X.XXXXXX.] duration: 363.131752ms FAIL (2025-09-22 02:14:46 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Sep 22 02:13:31.123370419 (564h5m6.476155547s) [upload=496426]: Starting upload Sep 22 02:13:31.123482970 (564h5m6.476268108s) [upload=496426]: Done starting upload Sep 22 02:13:31.123900823 (564h5m6.476685952s) [upload=496426]: Uploading segments... Sep 22 02:13:31.123976205 (564h5m6.476761333s) [upload=496426]: Got next segment. Inline: false Sep 22 02:13:31.124008105 (564h5m6.476793233s) [upload=496426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:13:31.124015108 (564h5m6.476800236s) [upload=496426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:13:31.148379950 (564h5m6.501165049s) [seg_pos={0 0}, upload=496426]: Begin upload segment... Sep 22 02:13:31.148424965 (564h5m6.501210063s) [seg_pos={0 0}, upload=496426]: Done begin upload segment. Sep 22 02:13:31.148467605 (564h5m6.501252703s) [noise=true , address=145.79.227.28:4729 , piece_id=3BFJAEVK4MDGJPLI5FQX7GTFVP7G5VE5EPVI4CKBG6TXNDRJPO6A, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.148650889 (564h5m6.501436016s) [noise=true , address=145.79.227.25:4651 , piece_id=XBVPDQH5NOTHMMQPBAHF5MIJTXVGU37LLJTF35HEXYEN6AEIMV3A, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.148844502 (564h5m6.50162961s) [noise=true , address=145.79.227.17:4997 , piece_id=BDNGRBTCNQK7AOFEHP4IUDTHLH2HPN2P6XYL25NPAYGAFKDIHZ6Q, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.149209627 (564h5m6.501994736s) [noise=true , address=145.79.227.30:4764 , piece_id=IW5TJOGT3AAKO2PBZXVW6PN3FUS4LM23VGVPYYUNNIQUE3XUBMXQ, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.149249272 (564h5m6.5020344s) [noise=true , address=145.79.227.46:4602 , piece_id=XNKJNNJASPEHCR3UVDTU55QFKVPNRWLDS3E3POQ7LUG5SX6IKUZA, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.149564203 (564h5m6.502349301s) [noise=true , address=145.79.227.12:4785 , piece_id=3LQDV6LTJBIKUMSHHJWOUCNVV2ZCLXBEYCVL4XFMUIN7374AIJBQ, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.149758056 (564h5m6.502543185s) [noise=true , address=145.79.227.45:4258 , piece_id=HJCI2PBCN27XBTXUJPYV7MADQKKKX5CB3TVAKXITEH3CCA7TZSVA, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.149834681 (564h5m6.502619779s) [noise=true , address=145.79.227.5:4236 , piece_id=KVODBMFFVHK2PUHPGVLREOV5SK7YE7B2IYXVECYDI655GUWBEUJQ, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.150103565 (564h5m6.502888674s) [noise=true , address=145.79.227.16:4915 , piece_id=77ESS2CVFF62METXPWC5NNB54L6MOPCGFTNYBY7AHJQRPW76CYLA, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.150228108 (564h5m6.503013237s) [noise=true , address=145.79.227.27:4716 , piece_id=QU2NFV24XNCXFCQAKKX32TRAVEGKBWCAZXQ2OZFNAEMV7M544I2A, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.150355278 (564h5m6.503140386s) [noise=true , address=145.79.227.35:4512 , piece_id=DZ3UZX2WCHGG2AIAI6QU6GDET42XYN3OQ72VT654ULTNHQ3WXZ7A, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.150621356 (564h5m6.503406464s) [noise=true , address=145.79.227.23:4556 , piece_id=4OJ2XWKI6RFRWKKTEK6AD2MHVHWFCFJ6GSM6P4TBJ3APXL4C6HEQ, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.150786868 (564h5m6.503571996s) [noise=true , address=145.79.227.26:4653 , piece_id=FLZGOCSUBA4DYJWANSNQFKAYD723ETVVQVZ65DBAYGZ6MEMU7BQQ, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.150893007 (564h5m6.503678104s) [noise=true , address=145.79.227.49:4347 , piece_id=RJ3WB565YG3WCEPEAS7JT557G3UKSFS3ZVOT7GIV2222PDVFYTPQ, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.151216654 (564h5m6.504001782s) [noise=true , address=145.79.227.16:4917 , piece_id=IJMWS6SWG45VPUAISPJVHKPA6FUS4QQMR5D3WCHW54XDHSEPM24A, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.163881771 (564h5m6.516666879s) [noise=true , address=145.79.227.52:4465 , piece_id=SVLLR6OMTR75SPEPG5ULYYHQRP5AFJHDOIF7ME5ZGLCMS6ZES43A, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.164384484 (564h5m6.517169592s) [noise=true , address=145.79.227.43:4167 , piece_id=WLBLSW6LWMGHKU5LWYLQLDRFOZXTIEHWUJDZMMZVEWATWUO5M32A, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.164728540 (564h5m6.517513639s) [noise=true , address=145.79.227.48:4336 , piece_id=EZZILAKKPJOIOLV23GBINEHTN752RNOC4WFS3PUSYNVW2IYVDQ3Q, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.221725048 (564h5m6.574510146s) [upload=496426]: Next returned nil segment Sep 22 02:13:31.221737140 (564h5m6.574522238s) [upload=496426]: Waiting for error group managing segments... Sep 22 02:13:31.236128126 (564h5m6.588913224s) [noise=true , address=145.79.227.27:4716 , piece_id=QU2NFV24XNCXFCQAKKX32TRAVEGKBWCAZXQ2OZFNAEMV7M544I2A, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.240486955 (564h5m6.593272073s) [noise=true , address=145.79.227.28:4729 , piece_id=3BFJAEVK4MDGJPLI5FQX7GTFVP7G5VE5EPVI4CKBG6TXNDRJPO6A, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.241190977 (564h5m6.593976074s) [noise=true , address=145.79.227.30:4764 , piece_id=IW5TJOGT3AAKO2PBZXVW6PN3FUS4LM23VGVPYYUNNIQUE3XUBMXQ, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.250713206 (564h5m6.603498324s) [noise=true , address=145.79.227.46:4602 , piece_id=XNKJNNJASPEHCR3UVDTU55QFKVPNRWLDS3E3POQ7LUG5SX6IKUZA, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.251333439 (564h5m6.604118558s) [noise=true , address=145.79.227.26:4653 , piece_id=FLZGOCSUBA4DYJWANSNQFKAYD723ETVVQVZ65DBAYGZ6MEMU7BQQ, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.251939377 (564h5m6.604724475s) [noise=true , address=145.79.227.23:4556 , piece_id=4OJ2XWKI6RFRWKKTEK6AD2MHVHWFCFJ6GSM6P4TBJ3APXL4C6HEQ, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.253130262 (564h5m6.605915359s) [noise=true , address=145.79.227.35:4512 , piece_id=DZ3UZX2WCHGG2AIAI6QU6GDET42XYN3OQ72VT654ULTNHQ3WXZ7A, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.258561514 (564h5m6.611346612s) [noise=true , address=145.79.227.45:4258 , piece_id=HJCI2PBCN27XBTXUJPYV7MADQKKKX5CB3TVAKXITEH3CCA7TZSVA, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.263589358 (564h5m6.616374487s) [noise=true , address=145.79.227.52:4465 , piece_id=SVLLR6OMTR75SPEPG5ULYYHQRP5AFJHDOIF7ME5ZGLCMS6ZES43A, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.267382377 (564h5m6.620167475s) [noise=true , address=145.79.227.25:4651 , piece_id=XBVPDQH5NOTHMMQPBAHF5MIJTXVGU37LLJTF35HEXYEN6AEIMV3A, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.270822481 (564h5m6.62360761s) [noise=true , address=145.79.227.48:4336 , piece_id=EZZILAKKPJOIOLV23GBINEHTN752RNOC4WFS3PUSYNVW2IYVDQ3Q, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.271707231 (564h5m6.62449236s) [noise=true , address=145.79.227.49:4347 , piece_id=RJ3WB565YG3WCEPEAS7JT557G3UKSFS3ZVOT7GIV2222PDVFYTPQ, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.272633100 (564h5m6.625418227s) [noise=true , address=145.79.227.43:4167 , piece_id=WLBLSW6LWMGHKU5LWYLQLDRFOZXTIEHWUJDZMMZVEWATWUO5M32A, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:14:31.101811916 (564h6m6.454597014s) [noise=true , address=145.79.227.16:4917 , piece_id=IJMWS6SWG45VPUAISPJVHKPA6FUS4QQMR5D3WCHW54XDHSEPM24A, seg_pos={0 0}, upload=496426]: Done uploading piece. err: ecclient: upload failed (node:1VaqXNoBYjwbbE9xhAoLzDuxZMdBPSm2aMw7bYm4qZqiEGp5CV, address:145.79.227.16:4917): protocol: rpc: dial tcp 145.79.227.16:4917: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:14:31.101898368 (564h6m6.454683476s) [noise=true , address=145.79.227.5:4236 , piece_id=KVODBMFFVHK2PUHPGVLREOV5SK7YE7B2IYXVECYDI655GUWBEUJQ, seg_pos={0 0}, upload=496426]: Done uploading piece. err: ecclient: upload failed (node:1XaE29W27ym2RmuGdBvHeKcQZZEUbask1zsm9Bz8xLokZNGfcj, address:145.79.227.5:4236): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:14:31.101937822 (564h6m6.454722919s) [noise=true , address=145.79.227.16:4915 , piece_id=77ESS2CVFF62METXPWC5NNB54L6MOPCGFTNYBY7AHJQRPW76CYLA, seg_pos={0 0}, upload=496426]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: rpc: dial tcp 145.79.227.16:4915: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:14:31.101956186 (564h6m6.454741294s) [noise=true , address=145.79.227.17:4997 , piece_id=BDNGRBTCNQK7AOFEHP4IUDTHLH2HPN2P6XYL25NPAYGAFKDIHZ6Q, seg_pos={0 0}, upload=496426]: Done uploading piece. err: ecclient: upload failed (node:12UrPGB76X7LaXe3ioiu4NtUQ5FwspKYtBWD6rcMmZC6xbNnCAF, address:145.79.227.17:4997): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:14:31.101961587 (564h6m6.454746695s) [noise=true , address=145.79.227.12:4785 , piece_id=3LQDV6LTJBIKUMSHHJWOUCNVV2ZCLXBEYCVL4XFMUIN7374AIJBQ, seg_pos={0 0}, upload=496426]: Done uploading piece. err: ecclient: upload failed (node:12u5zY7F7oy2tPUyFfoiebzFEySetBks9LC3gRGPURAyDDnsUu9, address:145.79.227.12:4785): protocol: rpc: dial tcp 145.79.227.12:4785: i/o timeout; context deadline exceeded Sep 22 02:14:31.101981153 (564h6m6.454766261s) [seg_pos={0 0}, upload=496426]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 02:14:31.118073471 (564h6m6.47085857s) [upload=496426]: Done uploading segments... Sep 22 02:14:31.118083610 (564h6m6.470868718s) [upload=496426]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 363.150498ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................................X.X.......] duration: 711.486402ms FAIL (2025-09-22 02:14:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 711.504376ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 7 out of 60 is failed (threshold=3) [....................................................X.XXXXXX] duration: 1m0.017573251s FAIL (2025-09-22 02:14:31 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Sep 22 02:13:31.123370419 (564h5m6.476155547s) [upload=496426]: Starting upload Sep 22 02:13:31.123482970 (564h5m6.476268108s) [upload=496426]: Done starting upload Sep 22 02:13:31.123900823 (564h5m6.476685952s) [upload=496426]: Uploading segments... Sep 22 02:13:31.123976205 (564h5m6.476761333s) [upload=496426]: Got next segment. Inline: false Sep 22 02:13:31.124008105 (564h5m6.476793233s) [upload=496426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:13:31.124015108 (564h5m6.476800236s) [upload=496426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:13:31.148379950 (564h5m6.501165049s) [seg_pos={0 0}, upload=496426]: Begin upload segment... Sep 22 02:13:31.148424965 (564h5m6.501210063s) [seg_pos={0 0}, upload=496426]: Done begin upload segment. Sep 22 02:13:31.148467605 (564h5m6.501252703s) [noise=true , address=145.79.227.28:4729 , piece_id=3BFJAEVK4MDGJPLI5FQX7GTFVP7G5VE5EPVI4CKBG6TXNDRJPO6A, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.148650889 (564h5m6.501436016s) [noise=true , address=145.79.227.25:4651 , piece_id=XBVPDQH5NOTHMMQPBAHF5MIJTXVGU37LLJTF35HEXYEN6AEIMV3A, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.148844502 (564h5m6.50162961s) [noise=true , address=145.79.227.17:4997 , piece_id=BDNGRBTCNQK7AOFEHP4IUDTHLH2HPN2P6XYL25NPAYGAFKDIHZ6Q, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.149209627 (564h5m6.501994736s) [noise=true , address=145.79.227.30:4764 , piece_id=IW5TJOGT3AAKO2PBZXVW6PN3FUS4LM23VGVPYYUNNIQUE3XUBMXQ, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.149249272 (564h5m6.5020344s) [noise=true , address=145.79.227.46:4602 , piece_id=XNKJNNJASPEHCR3UVDTU55QFKVPNRWLDS3E3POQ7LUG5SX6IKUZA, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.149564203 (564h5m6.502349301s) [noise=true , address=145.79.227.12:4785 , piece_id=3LQDV6LTJBIKUMSHHJWOUCNVV2ZCLXBEYCVL4XFMUIN7374AIJBQ, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.149758056 (564h5m6.502543185s) [noise=true , address=145.79.227.45:4258 , piece_id=HJCI2PBCN27XBTXUJPYV7MADQKKKX5CB3TVAKXITEH3CCA7TZSVA, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.149834681 (564h5m6.502619779s) [noise=true , address=145.79.227.5:4236 , piece_id=KVODBMFFVHK2PUHPGVLREOV5SK7YE7B2IYXVECYDI655GUWBEUJQ, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.150103565 (564h5m6.502888674s) [noise=true , address=145.79.227.16:4915 , piece_id=77ESS2CVFF62METXPWC5NNB54L6MOPCGFTNYBY7AHJQRPW76CYLA, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.150228108 (564h5m6.503013237s) [noise=true , address=145.79.227.27:4716 , piece_id=QU2NFV24XNCXFCQAKKX32TRAVEGKBWCAZXQ2OZFNAEMV7M544I2A, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.150355278 (564h5m6.503140386s) [noise=true , address=145.79.227.35:4512 , piece_id=DZ3UZX2WCHGG2AIAI6QU6GDET42XYN3OQ72VT654ULTNHQ3WXZ7A, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.150621356 (564h5m6.503406464s) [noise=true , address=145.79.227.23:4556 , piece_id=4OJ2XWKI6RFRWKKTEK6AD2MHVHWFCFJ6GSM6P4TBJ3APXL4C6HEQ, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.150786868 (564h5m6.503571996s) [noise=true , address=145.79.227.26:4653 , piece_id=FLZGOCSUBA4DYJWANSNQFKAYD723ETVVQVZ65DBAYGZ6MEMU7BQQ, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.150893007 (564h5m6.503678104s) [noise=true , address=145.79.227.49:4347 , piece_id=RJ3WB565YG3WCEPEAS7JT557G3UKSFS3ZVOT7GIV2222PDVFYTPQ, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.151216654 (564h5m6.504001782s) [noise=true , address=145.79.227.16:4917 , piece_id=IJMWS6SWG45VPUAISPJVHKPA6FUS4QQMR5D3WCHW54XDHSEPM24A, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.163881771 (564h5m6.516666879s) [noise=true , address=145.79.227.52:4465 , piece_id=SVLLR6OMTR75SPEPG5ULYYHQRP5AFJHDOIF7ME5ZGLCMS6ZES43A, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.164384484 (564h5m6.517169592s) [noise=true , address=145.79.227.43:4167 , piece_id=WLBLSW6LWMGHKU5LWYLQLDRFOZXTIEHWUJDZMMZVEWATWUO5M32A, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.164728540 (564h5m6.517513639s) [noise=true , address=145.79.227.48:4336 , piece_id=EZZILAKKPJOIOLV23GBINEHTN752RNOC4WFS3PUSYNVW2IYVDQ3Q, seg_pos={0 0}, upload=496426]: Uploading piece... Sep 22 02:13:31.221725048 (564h5m6.574510146s) [upload=496426]: Next returned nil segment Sep 22 02:13:31.221737140 (564h5m6.574522238s) [upload=496426]: Waiting for error group managing segments... Sep 22 02:13:31.236128126 (564h5m6.588913224s) [noise=true , address=145.79.227.27:4716 , piece_id=QU2NFV24XNCXFCQAKKX32TRAVEGKBWCAZXQ2OZFNAEMV7M544I2A, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.240486955 (564h5m6.593272073s) [noise=true , address=145.79.227.28:4729 , piece_id=3BFJAEVK4MDGJPLI5FQX7GTFVP7G5VE5EPVI4CKBG6TXNDRJPO6A, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.241190977 (564h5m6.593976074s) [noise=true , address=145.79.227.30:4764 , piece_id=IW5TJOGT3AAKO2PBZXVW6PN3FUS4LM23VGVPYYUNNIQUE3XUBMXQ, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.250713206 (564h5m6.603498324s) [noise=true , address=145.79.227.46:4602 , piece_id=XNKJNNJASPEHCR3UVDTU55QFKVPNRWLDS3E3POQ7LUG5SX6IKUZA, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.251333439 (564h5m6.604118558s) [noise=true , address=145.79.227.26:4653 , piece_id=FLZGOCSUBA4DYJWANSNQFKAYD723ETVVQVZ65DBAYGZ6MEMU7BQQ, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.251939377 (564h5m6.604724475s) [noise=true , address=145.79.227.23:4556 , piece_id=4OJ2XWKI6RFRWKKTEK6AD2MHVHWFCFJ6GSM6P4TBJ3APXL4C6HEQ, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.253130262 (564h5m6.605915359s) [noise=true , address=145.79.227.35:4512 , piece_id=DZ3UZX2WCHGG2AIAI6QU6GDET42XYN3OQ72VT654ULTNHQ3WXZ7A, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.258561514 (564h5m6.611346612s) [noise=true , address=145.79.227.45:4258 , piece_id=HJCI2PBCN27XBTXUJPYV7MADQKKKX5CB3TVAKXITEH3CCA7TZSVA, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.263589358 (564h5m6.616374487s) [noise=true , address=145.79.227.52:4465 , piece_id=SVLLR6OMTR75SPEPG5ULYYHQRP5AFJHDOIF7ME5ZGLCMS6ZES43A, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.267382377 (564h5m6.620167475s) [noise=true , address=145.79.227.25:4651 , piece_id=XBVPDQH5NOTHMMQPBAHF5MIJTXVGU37LLJTF35HEXYEN6AEIMV3A, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.270822481 (564h5m6.62360761s) [noise=true , address=145.79.227.48:4336 , piece_id=EZZILAKKPJOIOLV23GBINEHTN752RNOC4WFS3PUSYNVW2IYVDQ3Q, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.271707231 (564h5m6.62449236s) [noise=true , address=145.79.227.49:4347 , piece_id=RJ3WB565YG3WCEPEAS7JT557G3UKSFS3ZVOT7GIV2222PDVFYTPQ, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:13:31.272633100 (564h5m6.625418227s) [noise=true , address=145.79.227.43:4167 , piece_id=WLBLSW6LWMGHKU5LWYLQLDRFOZXTIEHWUJDZMMZVEWATWUO5M32A, seg_pos={0 0}, upload=496426]: Done uploading piece. err: <nil> Sep 22 02:14:31.101811916 (564h6m6.454597014s) [noise=true , address=145.79.227.16:4917 , piece_id=IJMWS6SWG45VPUAISPJVHKPA6FUS4QQMR5D3WCHW54XDHSEPM24A, seg_pos={0 0}, upload=496426]: Done uploading piece. err: ecclient: upload failed (node:1VaqXNoBYjwbbE9xhAoLzDuxZMdBPSm2aMw7bYm4qZqiEGp5CV, address:145.79.227.16:4917): protocol: rpc: dial tcp 145.79.227.16:4917: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:14:31.101898368 (564h6m6.454683476s) [noise=true , address=145.79.227.5:4236 , piece_id=KVODBMFFVHK2PUHPGVLREOV5SK7YE7B2IYXVECYDI655GUWBEUJQ, seg_pos={0 0}, upload=496426]: Done uploading piece. err: ecclient: upload failed (node:1XaE29W27ym2RmuGdBvHeKcQZZEUbask1zsm9Bz8xLokZNGfcj, address:145.79.227.5:4236): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:14:31.101937822 (564h6m6.454722919s) [noise=true , address=145.79.227.16:4915 , piece_id=77ESS2CVFF62METXPWC5NNB54L6MOPCGFTNYBY7AHJQRPW76CYLA, seg_pos={0 0}, upload=496426]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: rpc: dial tcp 145.79.227.16:4915: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:14:31.101956186 (564h6m6.454741294s) [noise=true , address=145.79.227.17:4997 , piece_id=BDNGRBTCNQK7AOFEHP4IUDTHLH2HPN2P6XYL25NPAYGAFKDIHZ6Q, seg_pos={0 0}, upload=496426]: Done uploading piece. err: ecclient: upload failed (node:12UrPGB76X7LaXe3ioiu4NtUQ5FwspKYtBWD6rcMmZC6xbNnCAF, address:145.79.227.17:4997): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:14:31.101961587 (564h6m6.454746695s) [noise=true , address=145.79.227.12:4785 , piece_id=3LQDV6LTJBIKUMSHHJWOUCNVV2ZCLXBEYCVL4XFMUIN7374AIJBQ, seg_pos={0 0}, upload=496426]: Done uploading piece. err: ecclient: upload failed (node:12u5zY7F7oy2tPUyFfoiebzFEySetBks9LC3gRGPURAyDDnsUu9, address:145.79.227.12:4785): protocol: rpc: dial tcp 145.79.227.12:4785: i/o timeout; context deadline exceeded Sep 22 02:14:31.101981153 (564h6m6.454766261s) [seg_pos={0 0}, upload=496426]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 02:14:31.118073471 (564h6m6.47085857s) [upload=496426]: Done uploading segments... Sep 22 02:14:31.118083610 (564h6m6.470868718s) [upload=496426]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.017582848s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.................................................XXX....XXX.] duration: 622.40318ms FAIL (2025-09-22 02:14:24 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:13:28.334561565 (564h5m3.687346663s) [upload=496423]: Starting upload Sep 22 02:13:28.334610847 (564h5m3.687395956s) [upload=496423]: Done starting upload Sep 22 02:13:28.334645232 (564h5m3.68743033s) [upload=496423]: Uploading segments... Sep 22 02:13:28.334823598 (564h5m3.687608706s) [upload=496423]: Got next segment. Inline: false Sep 22 02:13:28.334861228 (564h5m3.687646336s) [upload=496423]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:13:28.334864765 (564h5m3.687649873s) [upload=496423]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:13:28.357935879 (564h5m3.710720977s) [seg_pos={0 0}, upload=496423]: Begin upload segment... Sep 22 02:13:28.357962629 (564h5m3.710747737s) [seg_pos={0 0}, upload=496423]: Done begin upload segment. Sep 22 02:13:28.357981274 (564h5m3.710766382s) [noise=true , address=77.90.31.12:4592 , piece_id=SOKXLYYFOBR37BBH6LH6I3OIPS7YPHRRMUNLY2OSJZSVIJVIINZA, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.357989609 (564h5m3.710774708s) [noise=true , address=45.91.116.9:4263 , piece_id=FCUCQDGUNI5HBTEDD23YYE5I2FOFO744BAITIA4665RNXHGY6ATA, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.358273512 (564h5m3.71105861s) [noise=true , address=77.90.31.10:4517 , piece_id=35RUEK7XYURM2SGI2QJGXYR2Y6VHEKCIQWY2VUHGYDZSJODGANJA, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.358277199 (564h5m3.711062297s) [noise=true , address=145.79.227.5:4221 , piece_id=RZHWIHEIEEOYQH46LPBNLPV2UZ6X47J4STRQYHH2BPSBGE6ACKPA, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.358516329 (564h5m3.711301437s) [noise=true , address=145.79.227.3:4149 , piece_id=4W36EV5JAUFDANKZAOGMARS2SEEICGVJWE6ON54QGUGGUIXTX7VA, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.358522239 (564h5m3.711307338s) [noise=true , address=45.91.116.7:4010 , piece_id=SN6E6DFC3MZB7NXZPDVJEQA2KRXZP46NPTOD3WX4CN6KZD7VRWPA, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.358725621 (564h5m3.711510729s) [noise=true , address=45.91.116.13:4153 , piece_id=F3F3Y7A5MSEXXTRE5VDG5USQEY22MLEJKD74RF6WF24AKB6CAASA, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.358731613 (564h5m3.711516721s) [noise=true , address=145.79.227.13:4802 , piece_id=SOVFTHBFPBHLW4IBAUQSNQMPE6EHKXZDD5JGIGMRSNCOR3Q57CDA, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.358925966 (564h5m3.711711065s) [noise=true , address=145.79.227.18:4007 , piece_id=OLZK3OQUT6N5MB6U66KDDMEATFIIEAYPSV3IJFU4VP3IVW3EVQ2A, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.358940584 (564h5m3.711725682s) [noise=true , address=45.91.116.10:4083 , piece_id=HYJU5W5IQOXPNR34UTMHYO6UJVBUPWER7TV7L7STLHRU3FNUT2VQ, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.359154666 (564h5m3.711939774s) [noise=true , address=77.90.31.16:4735 , piece_id=OGUQDBXD4NAFS5VP2FPBEGUMWO3MFROEOAH7WG3PT55AC3TYSMYQ, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.359361444 (564h5m3.712146542s) [noise=true , address=45.91.116.14:4189 , piece_id=S6EF6ITWLEGYRI2OSVKZRCZZT4UU5COKDZAESENVTVX5P52SBHKQ, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.359614138 (564h5m3.712399246s) [noise=true , address=45.91.116.5:4298 , piece_id=QNU4L6LFLKZZITNWTOKQ7KGP7KCWFXTYCUIRQ7YQ7MUUZSXUQ2LQ, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.359830274 (564h5m3.712615372s) [noise=true , address=45.91.116.9:4262 , piece_id=EET4T563U4QBN4P4PBWZW5UUP2JJ2GOWICUMFLDTPJNQLYHGFFQA, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.360060867 (564h5m3.712845975s) [noise=true , address=145.79.227.5:4222 , piece_id=6L7RA27AMFCWCOLCMFS723QUHFTLSFOR6F6ELD3IAKONSK7ZIGNQ, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.360473892 (564h5m3.713259s) [noise=true , address=77.90.31.15:4697 , piece_id=P6ZCZQJ43PLB7U7YBX4SOTO2KPB24C5IDFILXNYTEUW7HM6VDFHQ, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.426376913 (564h5m3.77916201s) [upload=496423]: Next returned nil segment Sep 22 02:13:28.426386801 (564h5m3.779171899s) [upload=496423]: Waiting for error group managing segments... Sep 22 02:13:28.434238656 (564h5m3.787023764s) [noise=true , address=45.91.116.5:4298 , piece_id=QNU4L6LFLKZZITNWTOKQ7KGP7KCWFXTYCUIRQ7YQ7MUUZSXUQ2LQ, seg_pos={0 0}, upload=496423]: Done uploading piece. err: <nil> Sep 22 02:13:28.434539941 (564h5m3.787325039s) [noise=true , address=45.91.116.13:4153 , piece_id=F3F3Y7A5MSEXXTRE5VDG5USQEY22MLEJKD74RF6WF24AKB6CAASA, seg_pos={0 0}, upload=496423]: Done uploading piece. err: <nil> Sep 22 02:13:28.435316629 (564h5m3.788101737s) [noise=true , address=45.91.116.10:4083 , piece_id=HYJU5W5IQOXPNR34UTMHYO6UJVBUPWER7TV7L7STLHRU3FNUT2VQ, seg_pos={0 0}, upload=496423]: Done uploading piece. err: <nil> Sep 22 02:13:28.435391219 (564h5m3.788176317s) [noise=true , address=45.91.116.14:4189 , piece_id=S6EF6ITWLEGYRI2OSVKZRCZZT4UU5COKDZAESENVTVX5P52SBHKQ, seg_pos={0 0}, upload=496423]: Done uploading piece. err: <nil> Sep 22 02:13:28.435695530 (564h5m3.788480638s) [noise=true , address=77.90.31.15:4697 , piece_id=P6ZCZQJ43PLB7U7YBX4SOTO2KPB24C5IDFILXNYTEUW7HM6VDFHQ, seg_pos={0 0}, upload=496423]: Done uploading piece. err: <nil> Sep 22 02:13:28.435833619 (564h5m3.788618727s) [noise=true , address=45.91.116.7:4010 , piece_id=SN6E6DFC3MZB7NXZPDVJEQA2KRXZP46NPTOD3WX4CN6KZD7VRWPA, seg_pos={0 0}, upload=496423]: Done uploading piece. err: <nil> Sep 22 02:13:28.436466667 (564h5m3.789251765s) [noise=true , address=77.90.31.10:4517 , piece_id=35RUEK7XYURM2SGI2QJGXYR2Y6VHEKCIQWY2VUHGYDZSJODGANJA, seg_pos={0 0}, upload=496423]: Done uploading piece. err: <nil> Sep 22 02:13:28.436868451 (564h5m3.789653559s) [noise=true , address=45.91.116.9:4262 , piece_id=EET4T563U4QBN4P4PBWZW5UUP2JJ2GOWICUMFLDTPJNQLYHGFFQA, seg_pos={0 0}, upload=496423]: Done uploading piece. err: <nil> Sep 22 02:13:28.437291224 (564h5m3.790076332s) [noise=true , address=77.90.31.12:4592 , piece_id=SOKXLYYFOBR37BBH6LH6I3OIPS7YPHRRMUNLY2OSJZSVIJVIINZA, seg_pos={0 0}, upload=496423]: Done uploading piece. err: <nil> Sep 22 02:13:28.438143343 (564h5m3.790928451s) [noise=true , address=45.91.116.9:4263 , piece_id=FCUCQDGUNI5HBTEDD23YYE5I2FOFO744BAITIA4665RNXHGY6ATA, seg_pos={0 0}, upload=496423]: Done uploading piece. err: <nil> Sep 22 02:13:28.441347957 (564h5m3.794133065s) [noise=true , address=77.90.31.16:4735 , piece_id=OGUQDBXD4NAFS5VP2FPBEGUMWO3MFROEOAH7WG3PT55AC3TYSMYQ, seg_pos={0 0}, upload=496423]: Done uploading piece. err: <nil> Sep 22 02:13:58.321898021 (564h5m33.67468315s) [noise=true , address=145.79.227.3:4149 , piece_id=4W36EV5JAUFDANKZAOGMARS2SEEICGVJWE6ON54QGUGGUIXTX7VA, seg_pos={0 0}, upload=496423]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: rpc: dial tcp 145.79.227.3:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:13:58.322102195 (564h5m33.674887324s) [noise=true , address=145.79.227.13:4802 , piece_id=SOVFTHBFPBHLW4IBAUQSNQMPE6EHKXZDD5JGIGMRSNCOR3Q57CDA, seg_pos={0 0}, upload=496423]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:13:58.322166756 (564h5m33.674951885s) [noise=true , address=145.79.227.5:4222 , piece_id=6L7RA27AMFCWCOLCMFS723QUHFTLSFOR6F6ELD3IAKONSK7ZIGNQ, seg_pos={0 0}, upload=496423]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:13:58.322208825 (564h5m33.674993953s) [noise=true , address=145.79.227.18:4007 , piece_id=OLZK3OQUT6N5MB6U66KDDMEATFIIEAYPSV3IJFU4VP3IVW3EVQ2A, seg_pos={0 0}, upload=496423]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: rpc: dial tcp 145.79.227.18:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:13:58.322332627 (564h5m33.675117756s) [noise=true , address=145.79.227.5:4221 , piece_id=RZHWIHEIEEOYQH46LPBNLPV2UZ6X47J4STRQYHH2BPSBGE6ACKPA, seg_pos={0 0}, upload=496423]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: rpc: dial tcp 145.79.227.5:4221: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 02:13:58.322356282 (564h5m33.67514141s) [seg_pos={0 0}, upload=496423]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:13:58.348318721 (564h5m33.70110385s) [upload=496423]: Done uploading segments... Sep 22 02:13:58.348363787 (564h5m33.701148926s) [upload=496423]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 622.42966ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..................................................XXX....XXX] duration: 30.028923819s FAIL (2025-09-22 02:13:58 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:13:28.334561565 (564h5m3.687346663s) [upload=496423]: Starting upload Sep 22 02:13:28.334610847 (564h5m3.687395956s) [upload=496423]: Done starting upload Sep 22 02:13:28.334645232 (564h5m3.68743033s) [upload=496423]: Uploading segments... Sep 22 02:13:28.334823598 (564h5m3.687608706s) [upload=496423]: Got next segment. Inline: false Sep 22 02:13:28.334861228 (564h5m3.687646336s) [upload=496423]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:13:28.334864765 (564h5m3.687649873s) [upload=496423]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:13:28.357935879 (564h5m3.710720977s) [seg_pos={0 0}, upload=496423]: Begin upload segment... Sep 22 02:13:28.357962629 (564h5m3.710747737s) [seg_pos={0 0}, upload=496423]: Done begin upload segment. Sep 22 02:13:28.357981274 (564h5m3.710766382s) [noise=true , address=77.90.31.12:4592 , piece_id=SOKXLYYFOBR37BBH6LH6I3OIPS7YPHRRMUNLY2OSJZSVIJVIINZA, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.357989609 (564h5m3.710774708s) [noise=true , address=45.91.116.9:4263 , piece_id=FCUCQDGUNI5HBTEDD23YYE5I2FOFO744BAITIA4665RNXHGY6ATA, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.358273512 (564h5m3.71105861s) [noise=true , address=77.90.31.10:4517 , piece_id=35RUEK7XYURM2SGI2QJGXYR2Y6VHEKCIQWY2VUHGYDZSJODGANJA, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.358277199 (564h5m3.711062297s) [noise=true , address=145.79.227.5:4221 , piece_id=RZHWIHEIEEOYQH46LPBNLPV2UZ6X47J4STRQYHH2BPSBGE6ACKPA, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.358516329 (564h5m3.711301437s) [noise=true , address=145.79.227.3:4149 , piece_id=4W36EV5JAUFDANKZAOGMARS2SEEICGVJWE6ON54QGUGGUIXTX7VA, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.358522239 (564h5m3.711307338s) [noise=true , address=45.91.116.7:4010 , piece_id=SN6E6DFC3MZB7NXZPDVJEQA2KRXZP46NPTOD3WX4CN6KZD7VRWPA, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.358725621 (564h5m3.711510729s) [noise=true , address=45.91.116.13:4153 , piece_id=F3F3Y7A5MSEXXTRE5VDG5USQEY22MLEJKD74RF6WF24AKB6CAASA, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.358731613 (564h5m3.711516721s) [noise=true , address=145.79.227.13:4802 , piece_id=SOVFTHBFPBHLW4IBAUQSNQMPE6EHKXZDD5JGIGMRSNCOR3Q57CDA, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.358925966 (564h5m3.711711065s) [noise=true , address=145.79.227.18:4007 , piece_id=OLZK3OQUT6N5MB6U66KDDMEATFIIEAYPSV3IJFU4VP3IVW3EVQ2A, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.358940584 (564h5m3.711725682s) [noise=true , address=45.91.116.10:4083 , piece_id=HYJU5W5IQOXPNR34UTMHYO6UJVBUPWER7TV7L7STLHRU3FNUT2VQ, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.359154666 (564h5m3.711939774s) [noise=true , address=77.90.31.16:4735 , piece_id=OGUQDBXD4NAFS5VP2FPBEGUMWO3MFROEOAH7WG3PT55AC3TYSMYQ, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.359361444 (564h5m3.712146542s) [noise=true , address=45.91.116.14:4189 , piece_id=S6EF6ITWLEGYRI2OSVKZRCZZT4UU5COKDZAESENVTVX5P52SBHKQ, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.359614138 (564h5m3.712399246s) [noise=true , address=45.91.116.5:4298 , piece_id=QNU4L6LFLKZZITNWTOKQ7KGP7KCWFXTYCUIRQ7YQ7MUUZSXUQ2LQ, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.359830274 (564h5m3.712615372s) [noise=true , address=45.91.116.9:4262 , piece_id=EET4T563U4QBN4P4PBWZW5UUP2JJ2GOWICUMFLDTPJNQLYHGFFQA, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.360060867 (564h5m3.712845975s) [noise=true , address=145.79.227.5:4222 , piece_id=6L7RA27AMFCWCOLCMFS723QUHFTLSFOR6F6ELD3IAKONSK7ZIGNQ, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.360473892 (564h5m3.713259s) [noise=true , address=77.90.31.15:4697 , piece_id=P6ZCZQJ43PLB7U7YBX4SOTO2KPB24C5IDFILXNYTEUW7HM6VDFHQ, seg_pos={0 0}, upload=496423]: Uploading piece... Sep 22 02:13:28.426376913 (564h5m3.77916201s) [upload=496423]: Next returned nil segment Sep 22 02:13:28.426386801 (564h5m3.779171899s) [upload=496423]: Waiting for error group managing segments... Sep 22 02:13:28.434238656 (564h5m3.787023764s) [noise=true , address=45.91.116.5:4298 , piece_id=QNU4L6LFLKZZITNWTOKQ7KGP7KCWFXTYCUIRQ7YQ7MUUZSXUQ2LQ, seg_pos={0 0}, upload=496423]: Done uploading piece. err: <nil> Sep 22 02:13:28.434539941 (564h5m3.787325039s) [noise=true , address=45.91.116.13:4153 , piece_id=F3F3Y7A5MSEXXTRE5VDG5USQEY22MLEJKD74RF6WF24AKB6CAASA, seg_pos={0 0}, upload=496423]: Done uploading piece. err: <nil> Sep 22 02:13:28.435316629 (564h5m3.788101737s) [noise=true , address=45.91.116.10:4083 , piece_id=HYJU5W5IQOXPNR34UTMHYO6UJVBUPWER7TV7L7STLHRU3FNUT2VQ, seg_pos={0 0}, upload=496423]: Done uploading piece. err: <nil> Sep 22 02:13:28.435391219 (564h5m3.788176317s) [noise=true , address=45.91.116.14:4189 , piece_id=S6EF6ITWLEGYRI2OSVKZRCZZT4UU5COKDZAESENVTVX5P52SBHKQ, seg_pos={0 0}, upload=496423]: Done uploading piece. err: <nil> Sep 22 02:13:28.435695530 (564h5m3.788480638s) [noise=true , address=77.90.31.15:4697 , piece_id=P6ZCZQJ43PLB7U7YBX4SOTO2KPB24C5IDFILXNYTEUW7HM6VDFHQ, seg_pos={0 0}, upload=496423]: Done uploading piece. err: <nil> Sep 22 02:13:28.435833619 (564h5m3.788618727s) [noise=true , address=45.91.116.7:4010 , piece_id=SN6E6DFC3MZB7NXZPDVJEQA2KRXZP46NPTOD3WX4CN6KZD7VRWPA, seg_pos={0 0}, upload=496423]: Done uploading piece. err: <nil> Sep 22 02:13:28.436466667 (564h5m3.789251765s) [noise=true , address=77.90.31.10:4517 , piece_id=35RUEK7XYURM2SGI2QJGXYR2Y6VHEKCIQWY2VUHGYDZSJODGANJA, seg_pos={0 0}, upload=496423]: Done uploading piece. err: <nil> Sep 22 02:13:28.436868451 (564h5m3.789653559s) [noise=true , address=45.91.116.9:4262 , piece_id=EET4T563U4QBN4P4PBWZW5UUP2JJ2GOWICUMFLDTPJNQLYHGFFQA, seg_pos={0 0}, upload=496423]: Done uploading piece. err: <nil> Sep 22 02:13:28.437291224 (564h5m3.790076332s) [noise=true , address=77.90.31.12:4592 , piece_id=SOKXLYYFOBR37BBH6LH6I3OIPS7YPHRRMUNLY2OSJZSVIJVIINZA, seg_pos={0 0}, upload=496423]: Done uploading piece. err: <nil> Sep 22 02:13:28.438143343 (564h5m3.790928451s) [noise=true , address=45.91.116.9:4263 , piece_id=FCUCQDGUNI5HBTEDD23YYE5I2FOFO744BAITIA4665RNXHGY6ATA, seg_pos={0 0}, upload=496423]: Done uploading piece. err: <nil> Sep 22 02:13:28.441347957 (564h5m3.794133065s) [noise=true , address=77.90.31.16:4735 , piece_id=OGUQDBXD4NAFS5VP2FPBEGUMWO3MFROEOAH7WG3PT55AC3TYSMYQ, seg_pos={0 0}, upload=496423]: Done uploading piece. err: <nil> Sep 22 02:13:58.321898021 (564h5m33.67468315s) [noise=true , address=145.79.227.3:4149 , piece_id=4W36EV5JAUFDANKZAOGMARS2SEEICGVJWE6ON54QGUGGUIXTX7VA, seg_pos={0 0}, upload=496423]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: rpc: dial tcp 145.79.227.3:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:13:58.322102195 (564h5m33.674887324s) [noise=true , address=145.79.227.13:4802 , piece_id=SOVFTHBFPBHLW4IBAUQSNQMPE6EHKXZDD5JGIGMRSNCOR3Q57CDA, seg_pos={0 0}, upload=496423]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:13:58.322166756 (564h5m33.674951885s) [noise=true , address=145.79.227.5:4222 , piece_id=6L7RA27AMFCWCOLCMFS723QUHFTLSFOR6F6ELD3IAKONSK7ZIGNQ, seg_pos={0 0}, upload=496423]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:13:58.322208825 (564h5m33.674993953s) [noise=true , address=145.79.227.18:4007 , piece_id=OLZK3OQUT6N5MB6U66KDDMEATFIIEAYPSV3IJFU4VP3IVW3EVQ2A, seg_pos={0 0}, upload=496423]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: rpc: dial tcp 145.79.227.18:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:13:58.322332627 (564h5m33.675117756s) [noise=true , address=145.79.227.5:4221 , piece_id=RZHWIHEIEEOYQH46LPBNLPV2UZ6X47J4STRQYHH2BPSBGE6ACKPA, seg_pos={0 0}, upload=496423]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: rpc: dial tcp 145.79.227.5:4221: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 02:13:58.322356282 (564h5m33.67514141s) [seg_pos={0 0}, upload=496423]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:13:58.348318721 (564h5m33.70110385s) [upload=496423]: Done uploading segments... Sep 22 02:13:58.348363787 (564h5m33.701148926s) [upload=496423]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.028940981s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................................X.X......] duration: 439.663709ms FAIL (2025-09-22 02:13:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 439.684919ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....................................................X.XXXXX] duration: 1m0.013401513s FAIL (2025-09-22 02:13:31 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Sep 22 02:12:31.031728032 (564h4m6.38451314s) [upload=496411]: Starting upload Sep 22 02:12:31.031773607 (564h4m6.384558715s) [upload=496411]: Done starting upload Sep 22 02:12:31.031836075 (564h4m6.384621173s) [upload=496411]: Uploading segments... Sep 22 02:12:31.032011515 (564h4m6.384796613s) [upload=496411]: Got next segment. Inline: false Sep 22 02:12:31.032038215 (564h4m6.384823313s) [upload=496411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:12:31.032042743 (564h4m6.384827841s) [upload=496411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:12:31.058136190 (564h4m6.410921298s) [seg_pos={0 0}, upload=496411]: Begin upload segment... Sep 22 02:12:31.058170223 (564h4m6.410955331s) [seg_pos={0 0}, upload=496411]: Done begin upload segment. Sep 22 02:12:31.058187025 (564h4m6.410972133s) [noise=true , address=145.79.227.10:4403 , piece_id=ZF75FNP32GIKQ4CJJY4O7CXBBQHOO66R76NSRYNLHX6LLQG7LFIA, seg_pos={0 0}, upload=496411]: Uploading piece... Sep 22 02:12:31.058196022 (564h4m6.41098113s) [noise=true , address=77.90.31.11:4586 , piece_id=6H2CLPAN2RFDW44UMJW4YNJI3P42F5RZZGL66WV2RVIVOANABZKQ, seg_pos={0 0}, upload=496411]: Uploading piece... Sep 22 02:12:31.058236387 (564h4m6.411021486s) [noise=true , address=145.79.227.46:4634 , piece_id=YNXK2UPYI5OGF4VDUC5JXJOGJPFU3GLS2L4RUDAM6FNLXDAKGBOA, seg_pos={0 0}, upload=496411]: Uploading piece... Sep 22 02:12:31.058480806 (564h4m6.411265914s) [noise=true , address=145.79.227.5:4249 , piece_id=X55DK43WIGACPSHPN7RLCPGTIRXMNQLDHUCPNRETOAKZVDPWEHVA, seg_pos={0 0}, upload=496411]: Uploading piece... Sep 22 02:12:31.058484332 (564h4m6.41126944s) [noise=true , address=145.79.227.27:4689 , piece_id=KQRFJCNUW5OZS5MPKFGMCWNXGMTFMZWNN6YX5DE3VJQNOEMNQE6A, seg_pos={0 0}, upload=496411]: Uploading piece... Sep 22 02:12:31.058698845 (564h4m6.411483953s) [noise=true , address=45.91.116.8:4049 , piece_id=F7QQV4KIRFJKBXXT2THWDY7G5X5YIESK2CV3TBJRDYBJDECHYGIA, seg_pos={0 0}, upload=496411]: Uploading piece... Sep 22 02:12:31.058707081 (564h4m6.411492189s) [noise=true , address=145.79.227.39:4038 , piece_id=HFTCOXMUVE4QOG6RNYHN4VVRRAYVTWX2EAGKDGYOOJ3Y6KPODTJQ, seg_pos={0 0}, upload=496411]: Uploading piece... Sep 22 02:12:31.058939867 (564h4m6.411724965s) [noise=true , address=145.79.227.22:4538 , piece_id=HHTAWAFYN3LRF5LCRYNPELSC2UJ76OG5UW4G3HZHNEUJAORWRQGQ, seg_pos={0 0}, upload=496411]: Uploading piece... Sep 22 02:12:31.058944676 (564h4m6.411729784s) [noise=true , address=145.79.227.33:4883 , piece_id=IF25D42EHJ2VRRTRJO3DXNLJ3MUDYWM3KXI3Z3A66R7NUWU2WZTQ, seg_pos={0 0}, upload=496411]: Uploading piece... Sep 22 02:12:31.059170300 (564h4m6.411955408s) [noise=true , address=145.79.227.43:4169 , piece_id=YFHZFKO255E2N77SD6KDHYWR24ZAVZWG25GZNZWAD2RZBSWEPZ2A, seg_pos={0 0}, upload=496411]: Uploading piece... Sep 22 02:12:31.059176782 (564h4m6.41196189s) [noise=true , address=145.79.227.24:4590 , piece_id=MTJMEHTTRLQREABB5OQLZWCB4YEFMORFEKPQNL7EWJX7SMYCXPXA, seg_pos={0 0}, upload=496411]: Uploading piece... Sep 22 02:12:31.059384652 (564h4m6.41216976s) [noise=true , address=145.79.227.45:4247 , piece_id=YASBT62ET7VWW6MNXFAYVI2M37CU6UQHH53LC7PSIVQX4F4VXNUA, seg_pos={0 0}, upload=496411]: Uploading piece... Sep 22 02:12:31.059389782 (564h4m6.41217489s) [noise=true , address=145.79.227.13:4832 , piece_id=LGXEHDPR72S2776AC5Z2VKS4MN6XBEEZAFEMKDXYCABWPZJGYHUQ, seg_pos={0 0}, upload=496411]: Uploading piece... Sep 22 02:12:31.059609023 (564h4m6.412394121s) [noise=true , address=64.49.13.137:4499 , piece_id=J3XKXPEUNOZJHXQIW7VUSPFK6GRD4EO7J4TD5ZSVUIDZGDGB33JQ, seg_pos={0 0}, upload=496411]: Uploading piece... Sep 22 02:12:31.059809509 (564h4m6.412594607s) [noise=true , address=145.79.227.14:4862 , piece_id=SBHTVOIME24SGCFYOEYPTNZOFSNNGXZHODFPW6COBTTESEZTCW2Q, seg_pos={0 0}, upload=496411]: Uploading piece... Sep 22 02:12:31.060961131 (564h4m6.413746239s) [noise=true , address=145.79.227.23:4562 , piece_id=5EJKNPJUQNZDDSM54KJDYFLLKM4TZ32CC6CDWLZOQ6MAZ2BW7WLA, seg_pos={0 0}, upload=496411]: Uploading piece... Sep 22 02:12:31.061097287 (564h4m6.413882395s) [noise=true , address=145.79.227.31:4801 , piece_id=2IIY4MQMULYEC2QNLCETUZRL6GQHCRJPMXXFBFLS4RBXDDDQS32A, seg_pos={0 0}, upload=496411]: Uploading piece... Sep 22 02:12:31.061255994 (564h4m6.414041092s) [noise=true , address=145.79.227.49:4367 , piece_id=G5JTI65WSLXSTLSCIG76LSHEBWNYHJRKYVNJAOUDHDFW7MURAOWQ, seg_pos={0 0}, upload=496411]: Uploading piece... Sep 22 02:12:31.123968097 (564h4m6.476753205s) [upload=496411]: Next returned nil segment Sep 22 02:12:31.123979168 (564h4m6.476764266s) [upload=496411]: Waiting for error group managing segments... Sep 22 02:12:31.132228638 (564h4m6.485013727s) [noise=true , address=145.79.227.46:4634 , piece_id=YNXK2UPYI5OGF4VDUC5JXJOGJPFU3GLS2L4RUDAM6FNLXDAKGBOA, seg_pos={0 0}, upload=496411]: Done uploading piece. err: <nil> Sep 22 02:12:31.132817114 (564h4m6.485602222s) [noise=true , address=145.79.227.24:4590 , piece_id=MTJMEHTTRLQREABB5OQLZWCB4YEFMORFEKPQNL7EWJX7SMYCXPXA, seg_pos={0 0}, upload=496411]: Done uploading piece. err: <nil> Sep 22 02:12:31.133412700 (564h4m6.486197798s) [noise=true , address=77.90.31.11:4586 , piece_id=6H2CLPAN2RFDW44UMJW4YNJI3P42F5RZZGL66WV2RVIVOANABZKQ, seg_pos={0 0}, upload=496411]: Done uploading piece. err: <nil> Sep 22 02:12:31.133765654 (564h4m6.486550762s) [noise=true , address=145.79.227.49:4367 , piece_id=G5JTI65WSLXSTLSCIG76LSHEBWNYHJRKYVNJAOUDHDFW7MURAOWQ, seg_pos={0 0}, upload=496411]: Done uploading piece. err: <nil> Sep 22 02:12:31.133797443 (564h4m6.486582541s) [noise=true , address=145.79.227.23:4562 , piece_id=5EJKNPJUQNZDDSM54KJDYFLLKM4TZ32CC6CDWLZOQ6MAZ2BW7WLA, seg_pos={0 0}, upload=496411]: Done uploading piece. err: <nil> Sep 22 02:12:31.134116081 (564h4m6.486901179s) [noise=true , address=145.79.227.22:4538 , piece_id=HHTAWAFYN3LRF5LCRYNPELSC2UJ76OG5UW4G3HZHNEUJAORWRQGQ, seg_pos={0 0}, upload=496411]: Done uploading piece. err: <nil> Sep 22 02:12:31.134735062 (564h4m6.487520161s) [noise=true , address=145.79.227.33:4883 , piece_id=IF25D42EHJ2VRRTRJO3DXNLJ3MUDYWM3KXI3Z3A66R7NUWU2WZTQ, seg_pos={0 0}, upload=496411]: Done uploading piece. err: <nil> Sep 22 02:12:31.140475665 (564h4m6.493260763s) [noise=true , address=145.79.227.43:4169 , piece_id=YFHZFKO255E2N77SD6KDHYWR24ZAVZWG25GZNZWAD2RZBSWEPZ2A, seg_pos={0 0}, upload=496411]: Done uploading piece. err: <nil> Sep 22 02:12:31.143685809 (564h4m6.496470917s) [noise=true , address=145.79.227.27:4689 , piece_id=KQRFJCNUW5OZS5MPKFGMCWNXGMTFMZWNN6YX5DE3VJQNOEMNQE6A, seg_pos={0 0}, upload=496411]: Done uploading piece. err: <nil> Sep 22 02:12:31.144722855 (564h4m6.497507952s) [noise=true , address=45.91.116.8:4049 , piece_id=F7QQV4KIRFJKBXXT2THWDY7G5X5YIESK2CV3TBJRDYBJDECHYGIA, seg_pos={0 0}, upload=496411]: Done uploading piece. err: <nil> Sep 22 02:12:31.145509521 (564h4m6.498294619s) [noise=true , address=145.79.227.45:4247 , piece_id=YASBT62ET7VWW6MNXFAYVI2M37CU6UQHH53LC7PSIVQX4F4VXNUA, seg_pos={0 0}, upload=496411]: Done uploading piece. err: <nil> Sep 22 02:12:31.155600717 (564h4m6.508385816s) [noise=true , address=145.79.227.39:4038 , piece_id=HFTCOXMUVE4QOG6RNYHN4VVRRAYVTWX2EAGKDGYOOJ3Y6KPODTJQ, seg_pos={0 0}, upload=496411]: Done uploading piece. err: <nil> Sep 22 02:12:31.155692369 (564h4m6.508477477s) [noise=true , address=145.79.227.31:4801 , piece_id=2IIY4MQMULYEC2QNLCETUZRL6GQHCRJPMXXFBFLS4RBXDDDQS32A, seg_pos={0 0}, upload=496411]: Done uploading piece. err: <nil> Sep 22 02:13:31.017815358 (564h5m6.370600455s) [noise=true , address=64.49.13.137:4499 , piece_id=J3XKXPEUNOZJHXQIW7VUSPFK6GRD4EO7J4TD5ZSVUIDZGDGB33JQ, seg_pos={0 0}, upload=496411]: Done uploading piece. err: ecclient: upload failed (node:122jpykQPM6C83aeFAZh51h9DpL9iRp8wVfyovsdnL4HibA3JSt, address:64.49.13.137:4499): protocol: rpc: dial tcp 64.49.13.137:4499: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:13:31.017855643 (564h5m6.370640741s) [noise=true , address=145.79.227.10:4403 , piece_id=ZF75FNP32GIKQ4CJJY4O7CXBBQHOO66R76NSRYNLHX6LLQG7LFIA, seg_pos={0 0}, upload=496411]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: rpc: dial tcp 145.79.227.10:4403: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:13:31.017871032 (564h5m6.37065613s) [noise=true , address=145.79.227.13:4832 , piece_id=LGXEHDPR72S2776AC5Z2VKS4MN6XBEEZAFEMKDXYCABWPZJGYHUQ, seg_pos={0 0}, upload=496411]: Done uploading piece. err: ecclient: upload failed (node:1SWdNMwLYv7KiBqr5Pfo8qvxHnxirHGUDbhZWeojGXwueQUqr9, address:145.79.227.13:4832): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:13:31.017938679 (564h5m6.370723787s) [noise=true , address=145.79.227.14:4862 , piece_id=SBHTVOIME24SGCFYOEYPTNZOFSNNGXZHODFPW6COBTTESEZTCW2Q, seg_pos={0 0}, upload=496411]: Done uploading piece. err: ecclient: upload failed (node:12sUT7LBYFqWMUhx7MMjEmyiFDnbJnjSVndNbTaHioaVLLSXYue, address:145.79.227.14:4862): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:13:31.018124397 (564h5m6.370909495s) [noise=true , address=145.79.227.5:4249 , piece_id=X55DK43WIGACPSHPN7RLCPGTIRXMNQLDHUCPNRETOAKZVDPWEHVA, seg_pos={0 0}, upload=496411]: Done uploading piece. err: ecclient: upload failed (node:1GnfZTc54L9eRyjCUwQuaFcG6FjsUDjDwT3Gx4zoNTQrvP158L, address:145.79.227.5:4249): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:13:31.018190802 (564h5m6.3709759s) [seg_pos={0 0}, upload=496411]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 02:13:31.029784459 (564h5m6.382569567s) [upload=496411]: Done uploading segments... Sep 22 02:13:31.029804747 (564h5m6.382589855s) [upload=496411]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.013419055s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...................................................XXX....XX] duration: 30.016036813s FAIL (2025-09-22 02:13:28 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Sep 22 02:12:58.251339996 (564h4m33.604125125s) [upload=496418]: Starting upload Sep 22 02:12:58.251458539 (564h4m33.604243677s) [upload=496418]: Done starting upload Sep 22 02:12:58.251559127 (564h4m33.604344235s) [upload=496418]: Uploading segments... Sep 22 02:12:58.251875952 (564h4m33.60466105s) [upload=496418]: Got next segment. Inline: false Sep 22 02:12:58.251905417 (564h4m33.604690525s) [upload=496418]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:12:58.251932738 (564h4m33.604717837s) [upload=496418]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:12:58.278341126 (564h4m33.631126223s) [seg_pos={0 0}, upload=496418]: Begin upload segment... Sep 22 02:12:58.278364799 (564h4m33.631149907s) [seg_pos={0 0}, upload=496418]: Done begin upload segment. Sep 22 02:12:58.278381621 (564h4m33.631166719s) [noise=true , address=77.90.31.14:4663 , piece_id=LSKG2TTUIHZSPAMEMXNFS3PR3K77KQYAR24WR3AFZ55AIWUCR3KQ, seg_pos={0 0}, upload=496418]: Uploading piece... Sep 22 02:12:58.278538906 (564h4m33.631324034s) [noise=true , address=45.91.116.10:4083 , piece_id=HOY5XGSOA6MK5MXLVFRQEIII7F47UZF34KX6442NYWVS2JMUMLWA, seg_pos={0 0}, upload=496418]: Uploading piece... Sep 22 02:12:58.278664682 (564h4m33.63144979s) [noise=true , address=45.91.116.14:4190 , piece_id=DMUSH4KN3SWOTUE2V2GM3YOTIBPTROA376HGBAKYQDYDIF22Y6PQ, seg_pos={0 0}, upload=496418]: Uploading piece... Sep 22 02:12:58.278894775 (564h4m33.631679873s) [noise=true , address=145.79.227.16:4912 , piece_id=FP3SKTDPYKQBXMBOLH2F44OGPVOJIUZRULZOTR4645H2OQM2AXZQ, seg_pos={0 0}, upload=496418]: Uploading piece... Sep 22 02:12:58.279106131 (564h4m33.631891229s) [noise=true , address=145.79.227.19:4078 , piece_id=X6GA4RIMWHRCSR727PVCFXJZEYKRPPCO2XOM6F3BVR5DEHFNIJEQ, seg_pos={0 0}, upload=496418]: Uploading piece... Sep 22 02:12:58.279229442 (564h4m33.632014581s) [noise=true , address=145.79.227.18:4005 , piece_id=GOUXYAWBX46T6TBPAVYXMFYFPSGP5LUV2BAR5KMTAMBSBCAZSKRA, seg_pos={0 0}, upload=496418]: Uploading piece... Sep 22 02:12:58.279320703 (564h4m33.632105811s) [noise=true , address=45.91.116.5:4298 , piece_id=I3AQUN2AW2FYHBSTVW5S6ILOR5BQSIT7XNSGQQAP7TS3REXVKEEA, seg_pos={0 0}, upload=496418]: Uploading piece... Sep 22 02:12:58.279571694 (564h4m33.632356792s) [noise=true , address=145.79.227.17:4969 , piece_id=O7MODYGS44PBVR6DC6KYE5MEGT6RA3IX6KUG7TPDDA26EJZMREPQ, seg_pos={0 0}, upload=496418]: Uploading piece... Sep 22 02:12:58.279796186 (564h4m33.632581294s) [noise=true , address=145.79.227.17:4971 , piece_id=NM4RS5HYGF5HF5C76GDB5J2OC455E7PV4O4W4AQWZM44RQLELRPA, seg_pos={0 0}, upload=496418]: Uploading piece... Sep 22 02:12:58.279818839 (564h4m33.632604007s) [noise=true , address=45.91.116.9:4262 , piece_id=XY3FXL2RAMK3WWNLGDPURGBQJVA3OJ7GZJLQVPMI2VPLHUMOHNMQ, seg_pos={0 0}, upload=496418]: Uploading piece... Sep 22 02:12:58.280035325 (564h4m33.632820423s) [noise=true , address=45.91.116.14:4189 , piece_id=KEZVZUL4ED45M5HCDB7VRW7YTAYP2XGBE4MBRAMJVSRCINM3Q3AQ, seg_pos={0 0}, upload=496418]: Uploading piece... Sep 22 02:12:58.280377406 (564h4m33.633162535s) [noise=true , address=45.91.116.12:4227 , piece_id=7P4GB7FV6CS4QIR3TX42N7JYFGMTQEBFWMK3D6ILLZPMMZQ4EKOA, seg_pos={0 0}, upload=496418]: Uploading piece... Sep 22 02:12:58.280898224 (564h4m33.633683362s) [noise=true , address=145.79.227.15:4873 , piece_id=RTJKCAQDV4NS2JNB6U3ESFSTTFF3VODJG4M7MXFB2IQGV2KXP2EQ, seg_pos={0 0}, upload=496418]: Uploading piece... Sep 22 02:12:58.281513368 (564h4m33.634298506s) [noise=true , address=45.91.116.6:4974 , piece_id=G6BPGORAYNJRXZQUAILYTFEOU6HVPQDSACZUESUNMCC4JWZWEG3Q, seg_pos={0 0}, upload=496418]: Uploading piece... Sep 22 02:12:58.282114146 (564h4m33.634899274s) [noise=true , address=45.91.116.4:4937 , piece_id=C3NEZORGSSDSKGXSMOLKSKDSL6UQK5JYZWTPSWXXZNFTPDUZIHPQ, seg_pos={0 0}, upload=496418]: Uploading piece... Sep 22 02:12:58.282514858 (564h4m33.635299996s) [noise=true , address=145.79.227.11:4731 , piece_id=DJUU6AWR3VXERZ6MAIMM6DX2EYXW62A2AYM77EYFHGLLJNCMJNHQ, seg_pos={0 0}, upload=496418]: Uploading piece... Sep 22 02:12:58.377031816 (564h4m33.729816943s) [upload=496418]: Next returned nil segment Sep 22 02:12:58.377049949 (564h4m33.729835078s) [upload=496418]: Waiting for error group managing segments... Sep 22 02:12:58.386615410 (564h4m33.739400518s) [noise=true , address=45.91.116.4:4937 , piece_id=C3NEZORGSSDSKGXSMOLKSKDSL6UQK5JYZWTPSWXXZNFTPDUZIHPQ, seg_pos={0 0}, upload=496418]: Done uploading piece. err: <nil> Sep 22 02:12:58.387250501 (564h4m33.740035609s) [noise=true , address=45.91.116.10:4083 , piece_id=HOY5XGSOA6MK5MXLVFRQEIII7F47UZF34KX6442NYWVS2JMUMLWA, seg_pos={0 0}, upload=496418]: Done uploading piece. err: <nil> Sep 22 02:12:58.387301307 (564h4m33.740086435s) [noise=true , address=45.91.116.6:4974 , piece_id=G6BPGORAYNJRXZQUAILYTFEOU6HVPQDSACZUESUNMCC4JWZWEG3Q, seg_pos={0 0}, upload=496418]: Done uploading piece. err: <nil> Sep 22 02:12:58.387628451 (564h4m33.740413559s) [noise=true , address=45.91.116.5:4298 , piece_id=I3AQUN2AW2FYHBSTVW5S6ILOR5BQSIT7XNSGQQAP7TS3REXVKEEA, seg_pos={0 0}, upload=496418]: Done uploading piece. err: <nil> Sep 22 02:12:58.387876015 (564h4m33.740661143s) [noise=true , address=45.91.116.12:4227 , piece_id=7P4GB7FV6CS4QIR3TX42N7JYFGMTQEBFWMK3D6ILLZPMMZQ4EKOA, seg_pos={0 0}, upload=496418]: Done uploading piece. err: <nil> Sep 22 02:12:58.388196526 (564h4m33.740981634s) [noise=true , address=45.91.116.14:4190 , piece_id=DMUSH4KN3SWOTUE2V2GM3YOTIBPTROA376HGBAKYQDYDIF22Y6PQ, seg_pos={0 0}, upload=496418]: Done uploading piece. err: <nil> Sep 22 02:12:58.389106403 (564h4m33.741891542s) [noise=true , address=45.91.116.14:4189 , piece_id=KEZVZUL4ED45M5HCDB7VRW7YTAYP2XGBE4MBRAMJVSRCINM3Q3AQ, seg_pos={0 0}, upload=496418]: Done uploading piece. err: <nil> Sep 22 02:12:58.389534697 (564h4m33.742319806s) [noise=true , address=45.91.116.9:4262 , piece_id=XY3FXL2RAMK3WWNLGDPURGBQJVA3OJ7GZJLQVPMI2VPLHUMOHNMQ, seg_pos={0 0}, upload=496418]: Done uploading piece. err: <nil> Sep 22 02:12:58.392098819 (564h4m33.744883927s) [noise=true , address=77.90.31.14:4663 , piece_id=LSKG2TTUIHZSPAMEMXNFS3PR3K77KQYAR24WR3AFZ55AIWUCR3KQ, seg_pos={0 0}, upload=496418]: Done uploading piece. err: <nil> Sep 22 02:13:28.240621772 (564h5m3.593406869s) [noise=true , address=145.79.227.16:4912 , piece_id=FP3SKTDPYKQBXMBOLH2F44OGPVOJIUZRULZOTR4645H2OQM2AXZQ, seg_pos={0 0}, upload=496418]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:13:28.240664080 (564h5m3.593449179s) [noise=true , address=145.79.227.11:4731 , piece_id=DJUU6AWR3VXERZ6MAIMM6DX2EYXW62A2AYM77EYFHGLLJNCMJNHQ, seg_pos={0 0}, upload=496418]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:13:28.240706731 (564h5m3.593491828s) [noise=true , address=145.79.227.17:4971 , piece_id=NM4RS5HYGF5HF5C76GDB5J2OC455E7PV4O4W4AQWZM44RQLELRPA, seg_pos={0 0}, upload=496418]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: rpc: dial tcp 145.79.227.17:4971: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:13:28.240727950 (564h5m3.593513049s) [noise=true , address=145.79.227.18:4005 , piece_id=GOUXYAWBX46T6TBPAVYXMFYFPSGP5LUV2BAR5KMTAMBSBCAZSKRA, seg_pos={0 0}, upload=496418]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:13:28.240746896 (564h5m3.593531994s) [noise=true , address=145.79.227.15:4873 , piece_id=RTJKCAQDV4NS2JNB6U3ESFSTTFF3VODJG4M7MXFB2IQGV2KXP2EQ, seg_pos={0 0}, upload=496418]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:13:28.240764489 (564h5m3.593549587s) [noise=true , address=145.79.227.19:4078 , piece_id=X6GA4RIMWHRCSR727PVCFXJZEYKRPPCO2XOM6F3BVR5DEHFNIJEQ, seg_pos={0 0}, upload=496418]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:13:28.240789566 (564h5m3.593574674s) [noise=true , address=145.79.227.17:4969 , piece_id=O7MODYGS44PBVR6DC6KYE5MEGT6RA3IX6KUG7TPDDA26EJZMREPQ, seg_pos={0 0}, upload=496418]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:145.79.227.17:4969): protocol: rpc: dial tcp 145.79.227.17:4969: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:13:28.240810074 (564h5m3.593595182s) [seg_pos={0 0}, upload=496418]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 22 02:13:28.254752237 (564h5m3.607537335s) [upload=496418]: Done uploading segments... Sep 22 02:13:28.254775992 (564h5m3.60756109s) [upload=496418]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.016058322s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................................X.X.....] duration: 619.798192ms FAIL (2025-09-22 02:13:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 619.817178ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................X.X....] duration: 419.614554ms FAIL (2025-09-22 02:12:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 419.637768ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................................................XXX....X] duration: 30.016080335s FAIL (2025-09-22 02:12:36 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== Sep 22 02:12:06.761513987 (564h3m42.114299124s) [upload=496407]: Starting upload Sep 22 02:12:06.761648500 (564h3m42.114433638s) [upload=496407]: Done starting upload Sep 22 02:12:06.761743187 (564h3m42.114528315s) [upload=496407]: Uploading segments... Sep 22 02:12:06.762103062 (564h3m42.11488819s) [upload=496407]: Got next segment. Inline: false Sep 22 02:12:06.762174807 (564h3m42.114959935s) [upload=496407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:12:06.762184956 (564h3m42.114970094s) [upload=496407]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:12:06.797022761 (564h3m42.1498079s) [seg_pos={0 0}, upload=496407]: Begin upload segment... Sep 22 02:12:06.797064560 (564h3m42.149849688s) [seg_pos={0 0}, upload=496407]: Done begin upload segment. Sep 22 02:12:06.797094987 (564h3m42.149880115s) [noise=true , address=145.79.227.14:4838 , piece_id=5RHCG4CG73WS3J7MZXYL7LDFN43WICUOLE7ZOLHWP3DV4AQVGFKQ, seg_pos={0 0}, upload=496407]: Uploading piece... Sep 22 02:12:06.797110827 (564h3m42.149895955s) [noise=true , address=77.90.31.15:4698 , piece_id=AMPE2UHCIOJXW3BFRHZY3RDSJTCQRSHULU77IMSMC3YWZKHO52LA, seg_pos={0 0}, upload=496407]: Uploading piece... Sep 22 02:12:06.797602540 (564h3m42.150387668s) [noise=true , address=145.79.227.15:4874 , piece_id=Y46HCUA6KU5MLP2RKRG47XROHMA3RHJGOZHWHWIYS7NX4ZUN7KHQ, seg_pos={0 0}, upload=496407]: Uploading piece... Sep 22 02:12:06.797654457 (564h3m42.150439585s) [noise=true , address=145.79.227.13:4803 , piece_id=TPYZ6WW7SOJAZFGZO4L5RIPQ44IPBM47TFUJIC6RLNHUMUOYXWXQ, seg_pos={0 0}, upload=496407]: Uploading piece... Sep 22 02:12:06.797974808 (564h3m42.150759936s) [noise=true , address=45.91.116.7:4010 , piece_id=SBYYWHFZ5HTZLGN54UWRYA4OE7RIXOUNFIBA4JSAKAZCFC24KGZQ, seg_pos={0 0}, upload=496407]: Uploading piece... Sep 22 02:12:06.798034600 (564h3m42.150819728s) [noise=true , address=45.91.116.12:4225 , piece_id=NKD4ZSJBL7DPY2CC2EECOYD6RH6IMKHKKNBNRHVTW5KVSVAYIOTA, seg_pos={0 0}, upload=496407]: Uploading piece... Sep 22 02:12:06.798364849 (564h3m42.151149978s) [noise=true , address=145.79.227.18:4006 , piece_id=MZXCNGWS5P5MNRJFCC47UGM52W4EX7ORH7THG7KMZ5VM3Y5LH2AA, seg_pos={0 0}, upload=496407]: Uploading piece... Sep 22 02:12:06.798382923 (564h3m42.151168052s) [noise=true , address=77.90.31.12:4590 , piece_id=5OP2JAXLIA6O5LF2QLUQZGHDNA5YCEXDTSV64WFJBEHLK2UFBFGQ, seg_pos={0 0}, upload=496407]: Uploading piece... Sep 22 02:12:06.798760803 (564h3m42.151545931s) [noise=true , address=45.91.116.4:4938 , piece_id=X2SUVW67WRNJ5KCEMYV4LER2VWBYNJSV4UXNLZRR4MBVGRXMHCHA, seg_pos={0 0}, upload=496407]: Uploading piece... Sep 22 02:12:06.799135165 (564h3m42.151920294s) [noise=true , address=145.79.227.2:4113 , piece_id=O74KAEYWRZAWZAQ6X4QEMWAQVYYG67OVVPPI4XLPU2LRSOXMM3TA, seg_pos={0 0}, upload=496407]: Uploading piece... Sep 22 02:12:06.799570552 (564h3m42.15235568s) [noise=true , address=145.79.227.19:4078 , piece_id=VLJODTY3G7IIPR64J5DSJAKFAEYHADPN5FANSGVAPG5FQXA6U7LQ, seg_pos={0 0}, upload=496407]: Uploading piece... Sep 22 02:12:06.799976845 (564h3m42.152761973s) [noise=true , address=145.79.227.17:4971 , piece_id=T5XCTD5WJX4PHOLIKW4TBKGE6RVKJBEMBEFX3V4W5OWJPZY3YSDA, seg_pos={0 0}, upload=496407]: Uploading piece... Sep 22 02:12:06.800919483 (564h3m42.153704622s) [noise=true , address=45.91.116.9:4263 , piece_id=DAIYCYLQSFITK65OY6EZOFTNVDYVVZKOU6NDSTXKZZQF3BQLEWZQ, seg_pos={0 0}, upload=496407]: Uploading piece... Sep 22 02:12:06.801330234 (564h3m42.154115362s) [noise=true , address=77.90.31.21:4878 , piece_id=675CWEF4AVSFNE7RW6VYFZ5S2JQPPR54QBVSCRKNUCQ53W3CIKOA, seg_pos={0 0}, upload=496407]: Uploading piece... Sep 22 02:12:06.801745554 (564h3m42.154530682s) [noise=true , address=145.79.227.15:4876 , piece_id=DTY7MQ3MT3LLF7GVTYNBUHJOYLIOVUTMV7AFJDTH6VKL7PPWSNDQ, seg_pos={0 0}, upload=496407]: Uploading piece... Sep 22 02:12:06.802102875 (564h3m42.154888002s) [noise=true , address=45.91.116.9:4261 , piece_id=MACU4JICSC63DOWPIWHUJ5DP7XBZOSOLRMT7EU5KMRYUFZWEGHFA, seg_pos={0 0}, upload=496407]: Uploading piece... Sep 22 02:12:06.895024668 (564h3m42.247809766s) [upload=496407]: Next returned nil segment Sep 22 02:12:06.895039806 (564h3m42.247824914s) [upload=496407]: Waiting for error group managing segments... Sep 22 02:12:06.903410184 (564h3m42.256195282s) [noise=true , address=45.91.116.9:4263 , piece_id=DAIYCYLQSFITK65OY6EZOFTNVDYVVZKOU6NDSTXKZZQF3BQLEWZQ, seg_pos={0 0}, upload=496407]: Done uploading piece. err: <nil> Sep 22 02:12:06.904206538 (564h3m42.256991637s) [noise=true , address=45.91.116.12:4225 , piece_id=NKD4ZSJBL7DPY2CC2EECOYD6RH6IMKHKKNBNRHVTW5KVSVAYIOTA, seg_pos={0 0}, upload=496407]: Done uploading piece. err: <nil> Sep 22 02:12:06.904896423 (564h3m42.257681532s) [noise=true , address=45.91.116.9:4261 , piece_id=MACU4JICSC63DOWPIWHUJ5DP7XBZOSOLRMT7EU5KMRYUFZWEGHFA, seg_pos={0 0}, upload=496407]: Done uploading piece. err: <nil> Sep 22 02:12:06.905311662 (564h3m42.25809677s) [noise=true , address=45.91.116.4:4938 , piece_id=X2SUVW67WRNJ5KCEMYV4LER2VWBYNJSV4UXNLZRR4MBVGRXMHCHA, seg_pos={0 0}, upload=496407]: Done uploading piece. err: <nil> Sep 22 02:12:06.906775109 (564h3m42.259560217s) [noise=true , address=45.91.116.7:4010 , piece_id=SBYYWHFZ5HTZLGN54UWRYA4OE7RIXOUNFIBA4JSAKAZCFC24KGZQ, seg_pos={0 0}, upload=496407]: Done uploading piece. err: <nil> Sep 22 02:12:06.907124314 (564h3m42.259909422s) [noise=true , address=77.90.31.21:4878 , piece_id=675CWEF4AVSFNE7RW6VYFZ5S2JQPPR54QBVSCRKNUCQ53W3CIKOA, seg_pos={0 0}, upload=496407]: Done uploading piece. err: <nil> Sep 22 02:12:06.907560393 (564h3m42.260345501s) [noise=true , address=77.90.31.12:4590 , piece_id=5OP2JAXLIA6O5LF2QLUQZGHDNA5YCEXDTSV64WFJBEHLK2UFBFGQ, seg_pos={0 0}, upload=496407]: Done uploading piece. err: <nil> Sep 22 02:12:06.908585526 (564h3m42.261370624s) [noise=true , address=77.90.31.15:4698 , piece_id=AMPE2UHCIOJXW3BFRHZY3RDSJTCQRSHULU77IMSMC3YWZKHO52LA, seg_pos={0 0}, upload=496407]: Done uploading piece. err: <nil> Sep 22 02:12:36.742072252 (564h4m12.09485736s) [noise=true , address=145.79.227.2:4113 , piece_id=O74KAEYWRZAWZAQ6X4QEMWAQVYYG67OVVPPI4XLPU2LRSOXMM3TA, seg_pos={0 0}, upload=496407]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: rpc: dial tcp 145.79.227.2:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:36.742343561 (564h4m12.095128669s) [noise=true , address=145.79.227.17:4971 , piece_id=T5XCTD5WJX4PHOLIKW4TBKGE6RVKJBEMBEFX3V4W5OWJPZY3YSDA, seg_pos={0 0}, upload=496407]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: rpc: dial tcp 145.79.227.17:4971: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:36.742401940 (564h4m12.095187039s) [noise=true , address=145.79.227.13:4803 , piece_id=TPYZ6WW7SOJAZFGZO4L5RIPQ44IPBM47TFUJIC6RLNHUMUOYXWXQ, seg_pos={0 0}, upload=496407]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: rpc: dial tcp 145.79.227.13:4803: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:36.742461763 (564h4m12.095246871s) [noise=true , address=145.79.227.15:4876 , piece_id=DTY7MQ3MT3LLF7GVTYNBUHJOYLIOVUTMV7AFJDTH6VKL7PPWSNDQ, seg_pos={0 0}, upload=496407]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: rpc: dial tcp 145.79.227.15:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:36.742496949 (564h4m12.095282057s) [noise=true , address=145.79.227.18:4006 , piece_id=MZXCNGWS5P5MNRJFCC47UGM52W4EX7ORH7THG7KMZ5VM3Y5LH2AA, seg_pos={0 0}, upload=496407]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: rpc: dial tcp 145.79.227.18:4006: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:36.742509382 (564h4m12.09529449s) [noise=true , address=145.79.227.15:4874 , piece_id=Y46HCUA6KU5MLP2RKRG47XROHMA3RHJGOZHWHWIYS7NX4ZUN7KHQ, seg_pos={0 0}, upload=496407]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: rpc: dial tcp 145.79.227.15:4874: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:36.742545550 (564h4m12.095330658s) [noise=true , address=145.79.227.14:4838 , piece_id=5RHCG4CG73WS3J7MZXYL7LDFN43WICUOLE7ZOLHWP3DV4AQVGFKQ, seg_pos={0 0}, upload=496407]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: rpc: dial tcp 145.79.227.14:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:36.742554317 (564h4m12.095339425s) [noise=true , address=145.79.227.19:4078 , piece_id=VLJODTY3G7IIPR64J5DSJAKFAEYHADPN5FANSGVAPG5FQXA6U7LQ, seg_pos={0 0}, upload=496407]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:36.742582840 (564h4m12.095367948s) [seg_pos={0 0}, upload=496407]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Sep 22 02:12:36.756733064 (564h4m12.109518162s) [upload=496407]: Done uploading segments... Sep 22 02:12:36.756753983 (564h4m12.109539091s) [upload=496407]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 30.016100613s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [......................................................X.XXXX] duration: 1m0.025049229s FAIL (2025-09-22 02:12:27 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== :27.326034439 (564h3m2.678819537s) [noise=true , address=145.79.227.19:4087 , piece_id=63DS2DTV4NSEEDYSJ7B5WQG5TGRP46R7UDP7WJ6LOA5RMSHLN37Q, seg_pos={0 0}, upload=496400]: Uploading piece... Sep 22 02:11:27.326041362 (564h3m2.67882646s) [noise=true , address=77.90.31.10:4548 , piece_id=YMSZ2EEO3F763SMMQCXCIXQ5LEUDHLFEOE3PF5R5KSFMEKYITZUA, seg_pos={0 0}, upload=496400]: Uploading piece... Sep 22 02:11:27.326314464 (564h3m2.679099572s) [noise=true , address=145.79.227.38:4977 , piece_id=MTL2IT3DPXNLI55WTEWXT3EIJQOSSSMTA6HMM3KPFOQFZRXQTMFA, seg_pos={0 0}, upload=496400]: Uploading piece... Sep 22 02:11:27.326318652 (564h3m2.67910376s) [noise=true , address=145.79.227.16:4931 , piece_id=XW5O6JJEN2ALLXCVFYJOMFLNXVG5EEKEI6FIIKLECWUIR3VM4ZRQ, seg_pos={0 0}, upload=496400]: Uploading piece... Sep 22 02:11:27.326559895 (564h3m2.679345003s) [noise=true , address=145.79.227.29:4961 , piece_id=T3WHSIKYMRPGA4K4C7BAGDQKXNG62A5Y5CNH2TYDQMXKZ4WQFBNA, seg_pos={0 0}, upload=496400]: Uploading piece... Sep 22 02:11:27.326572529 (564h3m2.679357637s) [noise=true , address=145.79.227.15:4905 , piece_id=M5WNZIE5DN566NSDAGGYI54OF4LL33FFHMMEYW6OASWW5GVFXB3Q, seg_pos={0 0}, upload=496400]: Uploading piece... Sep 22 02:11:27.326781360 (564h3m2.679566468s) [noise=true , address=145.79.227.18:4025 , piece_id=7KFFOOQTLSFZZS3YUZ3V4TATFJTL3BGCQNLENX7WEWYLHQVMRH3Q, seg_pos={0 0}, upload=496400]: Uploading piece... Sep 22 02:11:27.326808842 (564h3m2.67959395s) [noise=true , address=145.79.227.14:4863 , piece_id=KSNZAIEDU3GCVSRSPFLU3N7ZWBBXZQPX6ANST6NEDTNYOLJINTGA, seg_pos={0 0}, upload=496400]: Uploading piece... Sep 22 02:11:27.327050125 (564h3m2.679835222s) [noise=true , address=145.79.227.13:4812 , piece_id=AFWFAYOW35AMJKWXSQJMWMYF5USSGKWWJD7RAS3BJE5UK7BD3MOQ, seg_pos={0 0}, upload=496400]: Uploading piece... Sep 22 02:11:27.327265499 (564h3m2.680050607s) [noise=true , address=145.79.227.28:4742 , piece_id=4NSIVBCGRQQXQU4X7WPQIGEOJRPNTWJFXCW6JZNZKEPB7HIUPDDA, seg_pos={0 0}, upload=496400]: Uploading piece... Sep 22 02:11:27.327492855 (564h3m2.680277953s) [noise=true , address=145.79.227.49:4351 , piece_id=VIRVGHCK24KC7LJJNUW6L2MAMDMVPADRNEEUIJJHMZ2GDMGPN56Q, seg_pos={0 0}, upload=496400]: Uploading piece... Sep 22 02:11:27.327609274 (564h3m2.680394372s) [noise=true , address=145.79.227.33:4879 , piece_id=MQETQHNMX7N3J5Y6RESAVEHCDQA64TVOEK5DRXFOH4CQKGDRTEAA, seg_pos={0 0}, upload=496400]: Uploading piece... Sep 22 02:11:27.327709212 (564h3m2.68049432s) [noise=true , address=145.79.227.23:4558 , piece_id=MTZZY5ZNSQEBWYF3ZEGZZM4KXBXY7D4VZSE2RMXGECSE737J5USQ, seg_pos={0 0}, upload=496400]: Uploading piece... Sep 22 02:11:27.327852049 (564h3m2.680637148s) [noise=true , address=145.79.227.7:4303 , piece_id=CG34QC34J3YS5GT4LK2YNNFUORMKTRX2X2HYBSHEPAIHMYZ3OB7Q, seg_pos={0 0}, upload=496400]: Uploading piece... Sep 22 02:11:27.327973177 (564h3m2.680758285s) [noise=true , address=145.79.227.5:4238 , piece_id=H2BIJLSFQ7H6JQF3S5BP7ZEPY5L2EQQ7JWPDCNEYFIVLUBO37ORA, seg_pos={0 0}, upload=496400]: Uploading piece... Sep 22 02:11:27.328213388 (564h3m2.680998496s) [noise=true , address=145.79.227.46:4614 , piece_id=JEZN4RRHXMY3Z4UOV6VULB6LUV5VWIAUO4F6B24CBUNJYQ6HIYMQ, seg_pos={0 0}, upload=496400]: Uploading piece... Sep 22 02:11:27.328421057 (564h3m2.681206154s) [noise=true , address=145.79.227.39:4013 , piece_id=UQBCACQR5XT52XT5HKZAJHVWGEFZFB4M6JYEDAGTPPKRQOX7XBUQ, seg_pos={0 0}, upload=496400]: Uploading piece... Sep 22 02:11:27.330036859 (564h3m2.682821958s) [noise=true , address=45.91.116.14:4204 , piece_id=Q2N6TSRLTFBP4EZGHMWLAAT2M7H26D4OXBSZ7UNBINE4J4LNURSQ, seg_pos={0 0}, upload=496400]: Uploading piece... Sep 22 02:11:27.393158040 (564h3m2.745943148s) [upload=496400]: Next returned nil segment Sep 22 02:11:27.393168821 (564h3m2.745953919s) [upload=496400]: Waiting for error group managing segments... Sep 22 02:11:27.400183473 (564h3m2.752968562s) [noise=true , address=145.79.227.46:4614 , piece_id=JEZN4RRHXMY3Z4UOV6VULB6LUV5VWIAUO4F6B24CBUNJYQ6HIYMQ, seg_pos={0 0}, upload=496400]: Done uploading piece. err: <nil> Sep 22 02:11:27.401888503 (564h3m2.754673602s) [noise=true , address=145.79.227.33:4879 , piece_id=MQETQHNMX7N3J5Y6RESAVEHCDQA64TVOEK5DRXFOH4CQKGDRTEAA, seg_pos={0 0}, upload=496400]: Done uploading piece. err: <nil> Sep 22 02:11:27.404430163 (564h3m2.757215261s) [noise=true , address=77.90.31.10:4548 , piece_id=YMSZ2EEO3F763SMMQCXCIXQ5LEUDHLFEOE3PF5R5KSFMEKYITZUA, seg_pos={0 0}, upload=496400]: Done uploading piece. err: <nil> Sep 22 02:11:27.412799709 (564h3m2.765584808s) [noise=true , address=45.91.116.14:4204 , piece_id=Q2N6TSRLTFBP4EZGHMWLAAT2M7H26D4OXBSZ7UNBINE4J4LNURSQ, seg_pos={0 0}, upload=496400]: Done uploading piece. err: <nil> Sep 22 02:11:27.414762272 (564h3m2.767547391s) [noise=true , address=145.79.227.49:4351 , piece_id=VIRVGHCK24KC7LJJNUW6L2MAMDMVPADRNEEUIJJHMZ2GDMGPN56Q, seg_pos={0 0}, upload=496400]: Done uploading piece. err: <nil> Sep 22 02:11:27.418727213 (564h3m2.771512321s) [noise=true , address=145.79.227.38:4977 , piece_id=MTL2IT3DPXNLI55WTEWXT3EIJQOSSSMTA6HMM3KPFOQFZRXQTMFA, seg_pos={0 0}, upload=496400]: Done uploading piece. err: <nil> Sep 22 02:11:27.422008321 (564h3m2.774793418s) [noise=true , address=145.79.227.23:4558 , piece_id=MTZZY5ZNSQEBWYF3ZEGZZM4KXBXY7D4VZSE2RMXGECSE737J5USQ, seg_pos={0 0}, upload=496400]: Done uploading piece. err: <nil> Sep 22 02:11:27.428196482 (564h3m2.78098158s) [noise=true , address=145.79.227.39:4013 , piece_id=UQBCACQR5XT52XT5HKZAJHVWGEFZFB4M6JYEDAGTPPKRQOX7XBUQ, seg_pos={0 0}, upload=496400]: Done uploading piece. err: <nil> Sep 22 02:11:27.433033820 (564h3m2.785818918s) [noise=true , address=145.79.227.28:4742 , piece_id=4NSIVBCGRQQXQU4X7WPQIGEOJRPNTWJFXCW6JZNZKEPB7HIUPDDA, seg_pos={0 0}, upload=496400]: Done uploading piece. err: <nil> Sep 22 02:11:27.436052023 (564h3m2.788837131s) [noise=true , address=145.79.227.29:4961 , piece_id=T3WHSIKYMRPGA4K4C7BAGDQKXNG62A5Y5CNH2TYDQMXKZ4WQFBNA, seg_pos={0 0}, upload=496400]: Done uploading piece. err: <nil> Sep 22 02:12:27.283958191 (564h4m2.636743319s) [noise=true , address=145.79.227.14:4863 , piece_id=KSNZAIEDU3GCVSRSPFLU3N7ZWBBXZQPX6ANST6NEDTNYOLJINTGA, seg_pos={0 0}, upload=496400]: Done uploading piece. err: ecclient: upload failed (node:12ApjPajPXNsWuwWvZ778CTxL33xrFi2cebEDBNHhFa52Nm2aqn, address:145.79.227.14:4863): protocol: rpc: dial tcp 145.79.227.14:4863: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:27.284007473 (564h4m2.636792602s) [noise=true , address=145.79.227.13:4812 , piece_id=AFWFAYOW35AMJKWXSQJMWMYF5USSGKWWJD7RAS3BJE5UK7BD3MOQ, seg_pos={0 0}, upload=496400]: Done uploading piece. err: ecclient: upload failed (node:12gTRhEnzLoKmQYRttr4hwiTbJ4y3yqMUS1Qg7F2yohb81sd5KV, address:145.79.227.13:4812): protocol: rpc: dial tcp 145.79.227.13:4812: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:27.284037510 (564h4m2.636822638s) [noise=true , address=145.79.227.7:4303 , piece_id=CG34QC34J3YS5GT4LK2YNNFUORMKTRX2X2HYBSHEPAIHMYZ3OB7Q, seg_pos={0 0}, upload=496400]: Done uploading piece. err: ecclient: upload failed (node:12Ec4Xvp3nGRAm9BivLLTp6AVkMoz1i2AgXQdfQaAmNPLHWD4Yv, address:145.79.227.7:4303): protocol: rpc: dial tcp 145.79.227.7:4303: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:27.284079088 (564h4m2.636864216s) [noise=true , address=145.79.227.5:4238 , piece_id=H2BIJLSFQ7H6JQF3S5BP7ZEPY5L2EQQ7JWPDCNEYFIVLUBO37ORA, seg_pos={0 0}, upload=496400]: Done uploading piece. err: ecclient: upload failed (node:122AVjaN3YaiABnUjuZwSb6apPDeszZAWvmzwtP47k1fxsKLD5V, address:145.79.227.5:4238): protocol: rpc: dial tcp 145.79.227.5:4238: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:27.284088516 (564h4m2.636873653s) [noise=true , address=145.79.227.18:4025 , piece_id=7KFFOOQTLSFZZS3YUZ3V4TATFJTL3BGCQNLENX7WEWYLHQVMRH3Q, seg_pos={0 0}, upload=496400]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:145.79.227.18:4025): protocol: rpc: dial tcp 145.79.227.18:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:27.284130895 (564h4m2.636916023s) [noise=true , address=145.79.227.19:4087 , piece_id=63DS2DTV4NSEEDYSJ7B5WQG5TGRP46R7UDP7WJ6LOA5RMSHLN37Q, seg_pos={0 0}, upload=496400]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: rpc: dial tcp 145.79.227.19:4087: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:27.284138078 (564h4m2.636923207s) [noise=true , address=145.79.227.16:4931 , piece_id=XW5O6JJEN2ALLXCVFYJOMFLNXVG5EEKEI6FIIKLECWUIR3VM4ZRQ, seg_pos={0 0}, upload=496400]: Done uploading piece. err: ecclient: upload failed (node:125rm4HueKsNdwL3eCTCSWoPseR2LQWcsE3jNPpxSfYQQorj9LL, address:145.79.227.16:4931): protocol: rpc: dial tcp 145.79.227.16:4931: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:27.284171271 (564h4m2.636956399s) [noise=true , address=145.79.227.15:4905 , piece_id=M5WNZIE5DN566NSDAGGYI54OF4LL33FFHMMEYW6OASWW5GVFXB3Q, seg_pos={0 0}, upload=496400]: Done uploading piece. err: ecclient: upload failed (node:1qAL8paQ8Z3A9VgPRQ2UUG2LytmYZKdz9hjLzK8w8fQmHiYf88, address:145.79.227.15:4905): protocol: rpc: dial tcp 145.79.227.15:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:27.284223900 (564h4m2.637009027s) [seg_pos={0 0}, upload=496400]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Sep 22 02:12:27.306370439 (564h4m2.659155568s) [upload=496400]: Done uploading segments... Sep 22 02:12:27.306386950 (564h4m2.659172079s) [upload=496400]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1m0.025062572s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................................X.X...] duration: 394.023842ms FAIL (2025-09-22 02:11:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 394.046174ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................................................X.XXX] duration: 1m0.02370234s FAIL (2025-09-22 02:11:27 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== :27.233823778 (564h2m2.586608906s) [noise=true , address=145.79.227.49:4340 , piece_id=WFXYCZGYLYZN2WA7WFVWCSZAFSHPL4PVFVSWZK7TPC4HVDBGTLPA, seg_pos={0 0}, upload=496383]: Uploading piece... Sep 22 02:10:27.233838456 (564h2m2.586623554s) [noise=true , address=145.79.227.26:4675 , piece_id=GPSBTQTEPZTJCRDIFLF3L2V2CS6SDTIWT6RWUH2OITVRPA2H2GDA, seg_pos={0 0}, upload=496383]: Uploading piece... Sep 22 02:10:27.234157325 (564h2m2.586942433s) [noise=true , address=145.79.227.28:4733 , piece_id=H3Q6JVENGEMMQDILBIOJ3Y7XMHWI3GRVUQ3AJPN6AGHNRER77F6Q, seg_pos={0 0}, upload=496383]: Uploading piece... Sep 22 02:10:27.234369212 (564h2m2.58715434s) [noise=true , address=145.79.227.4:4207 , piece_id=Q35YVPNXSLLBRJJVAF3JFAALX3CC4UQEPHQ6TROWQIZMD4AK7AVA, seg_pos={0 0}, upload=496383]: Uploading piece... Sep 22 02:10:27.234396924 (564h2m2.587182032s) [noise=true , address=145.79.227.11:4753 , piece_id=V7ME5IDBWMY3GLGGSV255K55O4VMW6Z2U45UAXFAHV7KMA3GPPVQ, seg_pos={0 0}, upload=496383]: Uploading piece... Sep 22 02:10:27.234637936 (564h2m2.587423044s) [noise=true , address=145.79.227.40:4080 , piece_id=4LTUCIU3CRGAMOMTOJSRFD5PHHEKJZODRKQT6OKUSJS4RU7AQQMQ, seg_pos={0 0}, upload=496383]: Uploading piece... Sep 22 02:10:27.234821059 (564h2m2.587606187s) [noise=true , address=77.90.31.18:4813 , piece_id=J5XG5M6BS7LSVRC7HI4PCOWI7AVGQCBNLKINAE4LIHLWJNOVS6ZA, seg_pos={0 0}, upload=496383]: Uploading piece... Sep 22 02:10:27.234854323 (564h2m2.587639431s) [noise=true , address=145.79.227.39:4046 , piece_id=5ZW37VZ36DSDOQMNIBUABR2JUA4VEBS4X3THVMNHY2ADUOOJQ7MQ, seg_pos={0 0}, upload=496383]: Uploading piece... Sep 22 02:10:27.235251949 (564h2m2.588037076s) [noise=true , address=145.79.227.44:4226 , piece_id=NIZXTPSUZWFAXORVMRDFCHA7CLCDC2A3KOM34VCHHHR3TDNGMCQA, seg_pos={0 0}, upload=496383]: Uploading piece... Sep 22 02:10:27.235674331 (564h2m2.588459459s) [noise=true , address=145.79.227.12:4783 , piece_id=PKFMRLFHUJY3KMIR3D2TYJARQCX5JEOAR3RU3GZU5YEOTL7BGRBA, seg_pos={0 0}, upload=496383]: Uploading piece... Sep 22 02:10:27.236081435 (564h2m2.588866564s) [noise=true , address=145.79.227.14:4859 , piece_id=Y3PAA5ZFNDER44HAF6MKVLXBHHUTJBMQS5MUBTZDZTD4AFO25CDA, seg_pos={0 0}, upload=496383]: Uploading piece... Sep 22 02:10:27.236502525 (564h2m2.589287653s) [noise=true , address=145.79.227.20:4462 , piece_id=GNXRG5LN65EOEVZNHVXFBV4VA5BBWNU65QO2DPLUF2U5PK5ZBUUQ, seg_pos={0 0}, upload=496383]: Uploading piece... Sep 22 02:10:27.236881046 (564h2m2.589666174s) [noise=true , address=145.79.227.2:4127 , piece_id=RPI44BN5422MA2A52TY2U3T44U5LVGNHW6NSA6AWYI5PYOWC6PBQ, seg_pos={0 0}, upload=496383]: Uploading piece... Sep 22 02:10:27.237315461 (564h2m2.590100588s) [noise=true , address=145.79.227.17:4976 , piece_id=6TZD4KE74MVP7MBSZC76WCJTYWCAZAHYVPYBW2G5K56CS6NOPWEA, seg_pos={0 0}, upload=496383]: Uploading piece... Sep 22 02:10:27.237743695 (564h2m2.590528823s) [noise=true , address=145.79.227.25:4631 , piece_id=XK7TEKJCY642UPXHU5XYKOTVAVWKKCHKKACW5HVCMETT2DS2RSXA, seg_pos={0 0}, upload=496383]: Uploading piece... Sep 22 02:10:27.238155808 (564h2m2.590940936s) [noise=true , address=145.79.227.18:4008 , piece_id=ANOHPYKQTFHDHEZMAXLQ5SHUKAVE7JDYIZXQ2BKCSQZX3Y2N4SCA, seg_pos={0 0}, upload=496383]: Uploading piece... Sep 22 02:10:27.238575756 (564h2m2.591360895s) [noise=true , address=145.79.227.29:4947 , piece_id=YMZAYXTQKN6VTMUCFWHWMPV6IELTCCR3WAMXS4SNCDY4GK3TUMXA, seg_pos={0 0}, upload=496383]: Uploading piece... Sep 22 02:10:27.239009770 (564h2m2.591794899s) [noise=true , address=145.79.227.16:4920 , piece_id=PK6LOFHUAEWWKOFNUNVCHQLMI2NB3C6GFUSCMN2K6KFJ4Z64AVOA, seg_pos={0 0}, upload=496383]: Uploading piece... Sep 22 02:10:27.317524789 (564h2m2.670309897s) [upload=496383]: Next returned nil segment Sep 22 02:10:27.317539918 (564h2m2.670325026s) [upload=496383]: Waiting for error group managing segments... Sep 22 02:10:27.326325364 (564h2m2.679110491s) [noise=true , address=145.79.227.28:4733 , piece_id=H3Q6JVENGEMMQDILBIOJ3Y7XMHWI3GRVUQ3AJPN6AGHNRER77F6Q, seg_pos={0 0}, upload=496383]: Done uploading piece. err: <nil> Sep 22 02:10:27.328071530 (564h2m2.680856639s) [noise=true , address=145.79.227.26:4675 , piece_id=GPSBTQTEPZTJCRDIFLF3L2V2CS6SDTIWT6RWUH2OITVRPA2H2GDA, seg_pos={0 0}, upload=496383]: Done uploading piece. err: <nil> Sep 22 02:10:27.328408582 (564h2m2.681193701s) [noise=true , address=77.90.31.18:4813 , piece_id=J5XG5M6BS7LSVRC7HI4PCOWI7AVGQCBNLKINAE4LIHLWJNOVS6ZA, seg_pos={0 0}, upload=496383]: Done uploading piece. err: <nil> Sep 22 02:10:27.328528318 (564h2m2.681313426s) [noise=true , address=145.79.227.40:4080 , piece_id=4LTUCIU3CRGAMOMTOJSRFD5PHHEKJZODRKQT6OKUSJS4RU7AQQMQ, seg_pos={0 0}, upload=496383]: Done uploading piece. err: <nil> Sep 22 02:10:27.329927734 (564h2m2.682712842s) [noise=true , address=145.79.227.44:4226 , piece_id=NIZXTPSUZWFAXORVMRDFCHA7CLCDC2A3KOM34VCHHHR3TDNGMCQA, seg_pos={0 0}, upload=496383]: Done uploading piece. err: <nil> Sep 22 02:10:27.339998943 (564h2m2.692784051s) [noise=true , address=145.79.227.49:4340 , piece_id=WFXYCZGYLYZN2WA7WFVWCSZAFSHPL4PVFVSWZK7TPC4HVDBGTLPA, seg_pos={0 0}, upload=496383]: Done uploading piece. err: <nil> Sep 22 02:10:27.354014193 (564h2m2.706799301s) [noise=true , address=145.79.227.29:4947 , piece_id=YMZAYXTQKN6VTMUCFWHWMPV6IELTCCR3WAMXS4SNCDY4GK3TUMXA, seg_pos={0 0}, upload=496383]: Done uploading piece. err: <nil> Sep 22 02:10:27.361973520 (564h2m2.714758628s) [noise=true , address=145.79.227.25:4631 , piece_id=XK7TEKJCY642UPXHU5XYKOTVAVWKKCHKKACW5HVCMETT2DS2RSXA, seg_pos={0 0}, upload=496383]: Done uploading piece. err: <nil> Sep 22 02:10:27.362201147 (564h2m2.714986275s) [noise=true , address=145.79.227.39:4046 , piece_id=5ZW37VZ36DSDOQMNIBUABR2JUA4VEBS4X3THVMNHY2ADUOOJQ7MQ, seg_pos={0 0}, upload=496383]: Done uploading piece. err: <nil> Sep 22 02:10:27.373047069 (564h2m2.725832177s) [noise=true , address=145.79.227.20:4462 , piece_id=GNXRG5LN65EOEVZNHVXFBV4VA5BBWNU65QO2DPLUF2U5PK5ZBUUQ, seg_pos={0 0}, upload=496383]: Done uploading piece. err: <nil> Sep 22 02:11:27.193386155 (564h3m2.546171263s) [noise=true , address=145.79.227.4:4207 , piece_id=Q35YVPNXSLLBRJJVAF3JFAALX3CC4UQEPHQ6TROWQIZMD4AK7AVA, seg_pos={0 0}, upload=496383]: Done uploading piece. err: ecclient: upload failed (node:1bS1oz1rMxh8k1ZN4PALDgWTWw61uGGiqDX3gbXq49ehbDSFjw, address:145.79.227.4:4207): protocol: rpc: dial tcp 145.79.227.4:4207: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:11:27.193428724 (564h3m2.546213832s) [noise=true , address=145.79.227.17:4976 , piece_id=6TZD4KE74MVP7MBSZC76WCJTYWCAZAHYVPYBW2G5K56CS6NOPWEA, seg_pos={0 0}, upload=496383]: Done uploading piece. err: ecclient: upload failed (node:129av1sWniou1aWC1EReoKyaxztX2g6bQwzSFpGqwxx3ZQ3nz5e, address:145.79.227.17:4976): protocol: rpc: dial tcp 145.79.227.17:4976: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:11:27.193488376 (564h3m2.546273475s) [noise=true , address=145.79.227.11:4753 , piece_id=V7ME5IDBWMY3GLGGSV255K55O4VMW6Z2U45UAXFAHV7KMA3GPPVQ, seg_pos={0 0}, upload=496383]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:11:27.193516469 (564h3m2.546301577s) [noise=true , address=145.79.227.12:4783 , piece_id=PKFMRLFHUJY3KMIR3D2TYJARQCX5JEOAR3RU3GZU5YEOTL7BGRBA, seg_pos={0 0}, upload=496383]: Done uploading piece. err: ecclient: upload failed (node:12ito72D7vikRdjK7BPt4LnhAK84bzi8AbZZPNWrTqmbjwQT1N, address:145.79.227.12:4783): protocol: rpc: dial tcp 145.79.227.12:4783: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:11:27.193532509 (564h3m2.546317617s) [noise=true , address=145.79.227.14:4859 , piece_id=Y3PAA5ZFNDER44HAF6MKVLXBHHUTJBMQS5MUBTZDZTD4AFO25CDA, seg_pos={0 0}, upload=496383]: Done uploading piece. err: ecclient: upload failed (node:1EiYA6gHiLyHsMfPevj2gLqmqttu1knLw3AM8SbWWtgKEf7Csu, address:145.79.227.14:4859): protocol: rpc: dial tcp 145.79.227.14:4859: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:11:27.193563769 (564h3m2.546348867s) [noise=true , address=145.79.227.2:4127 , piece_id=RPI44BN5422MA2A52TY2U3T44U5LVGNHW6NSA6AWYI5PYOWC6PBQ, seg_pos={0 0}, upload=496383]: Done uploading piece. err: ecclient: upload failed (node:12Pxos2xcjHwcYyQnXeRL3PL72G9pXaAnBs8XnkVFkxegDRPrqp, address:145.79.227.2:4127): protocol: rpc: dial tcp 145.79.227.2:4127: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:11:27.193568337 (564h3m2.546353445s) [noise=true , address=145.79.227.18:4008 , piece_id=ANOHPYKQTFHDHEZMAXLQ5SHUKAVE7JDYIZXQ2BKCSQZX3Y2N4SCA, seg_pos={0 0}, upload=496383]: Done uploading piece. err: ecclient: upload failed (node:148CqQmrsJSfoGByHYRPoZmvP1VgQQpaMVLb7RgRQvNBuo7air, address:145.79.227.18:4008): protocol: rpc: dial tcp 145.79.227.18:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:11:27.193585379 (564h3m2.546370487s) [noise=true , address=145.79.227.16:4920 , piece_id=PK6LOFHUAEWWKOFNUNVCHQLMI2NB3C6GFUSCMN2K6KFJ4Z64AVOA, seg_pos={0 0}, upload=496383]: Done uploading piece. err: ecclient: upload failed (node:1vBccAgciQwbLaMMe9QKfYM3zxYTHqG3pfYQ7nh8eETr8JUskb, address:145.79.227.16:4920): protocol: rpc: dial tcp 145.79.227.16:4920: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:11:27.193597823 (564h3m2.546382931s) [seg_pos={0 0}, upload=496383]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Sep 22 02:11:27.215047685 (564h3m2.567832793s) [upload=496383]: Done uploading segments... Sep 22 02:11:27.215062062 (564h3m2.56784717s) [upload=496383]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1m0.023714973s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................................................XXX....] duration: 431.807385ms FAIL (2025-09-22 02:11:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:08:20.420565373 (563h59m55.773350502s) [upload=496352]: Starting upload Sep 22 02:08:20.420650714 (563h59m55.773435842s) [upload=496352]: Done starting upload Sep 22 02:08:20.420786879 (563h59m55.773572007s) [upload=496352]: Uploading segments... Sep 22 02:08:20.421010398 (563h59m55.773795526s) [upload=496352]: Got next segment. Inline: false Sep 22 02:08:20.421056685 (563h59m55.773841813s) [upload=496352]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:08:20.421061874 (563h59m55.773847003s) [upload=496352]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:08:20.460287685 (563h59m55.813072813s) [seg_pos={0 0}, upload=496352]: Begin upload segment... Sep 22 02:08:20.460321128 (563h59m55.813106256s) [seg_pos={0 0}, upload=496352]: Done begin upload segment. Sep 22 02:08:20.460346526 (563h59m55.813131653s) [noise=true , address=45.91.116.5:4299 , piece_id=7RLKVP75JC53US6SNGULMLNL2VQSBS4C2JXJ2XEJM5FWQN3JKW6A, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.460355883 (563h59m55.813141011s) [noise=true , address=145.79.227.7:4295 , piece_id=WMEUSUFV3UET3KJIHNKXSJOIBSWFX7YV2F5Z4PN53WG24TSVYKCQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.460801099 (563h59m55.813586227s) [noise=true , address=145.79.227.11:4731 , piece_id=IOSKWC6JD4IBOLF2IB443K522B67ET7CAG33VGXXQOMJAUFVZT5A, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.460815526 (563h59m55.813600654s) [noise=true , address=77.90.31.14:4663 , piece_id=6IB3E74QRO2AMLSZ3SOL6FHHCIAD2YCLSB327HX7JSSNMJHZ4YNQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461169329 (563h59m55.813954458s) [noise=true , address=77.90.31.10:4518 , piece_id=GSZBFIOCXFVRZNCDBC2DFJUJLYHWYW7MUAS7C4XINUX7IH5KUQDQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461183325 (563h59m55.813968464s) [noise=true , address=145.79.227.16:4909 , piece_id=P3Z2HGUTNOXDJF4TDR4LGXVWKFKUODFMSMHQGWT4NU3C5UYGXBVQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461597703 (563h59m55.814382831s) [noise=true , address=77.90.31.19:4842 , piece_id=YESLECXUJ5I2NOZXRW6HSET7YNWF6373QNIXCPHWMZHC5TFTYLJA, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461608854 (563h59m55.814393982s) [noise=true , address=45.91.116.9:4262 , piece_id=NM4FUTBS3RJWOHEVN4KMBL4TEEVBIOOSROBAWZWJKLQTWWJMGIBA, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461960725 (563h59m55.814745853s) [noise=true , address=45.91.116.4:4937 , piece_id=TSG3DWXGOAMIKEFOI3GL266ONTV5O5PZVJIJEJVA3GHZVEFLCWBQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461986163 (563h59m55.81477129s) [noise=true , address=145.79.227.15:4876 , piece_id=XPFBENRLP7GSVALZDUMBAIM5Y4S4MS7LGXLNGMNO3RZMLHSFU3GQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.462326511 (563h59m55.815111639s) [noise=true , address=77.90.31.17:4770 , piece_id=LBBHDCEYUGA5IL3SETBBBXN44RR3MPS6CET4WOVHEHBHIRLSOFSA, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.462337151 (563h59m55.815122279s) [noise=true , address=45.91.116.8:4048 , piece_id=KDUUMIWPL56B4KUYAHZ3SVL4E3EHYGB2NCSJKZESBPYNBCABUR7A, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.462702948 (563h59m55.815488076s) [noise=true , address=145.79.227.18:4007 , piece_id=X2ILLKTIZSULE6EL2DKZ2V7WZFWCXZ5Z7NMEQQU7ORG6QANYGE5A, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.463054467 (563h59m55.815839596s) [noise=true , address=45.91.116.4:4939 , piece_id=ZVT77KYOPEZU5NVKQZNJRLJK6AFBI2IGK77KSGRG4WS26GKPC5AA, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.463414943 (563h59m55.816200081s) [noise=true , address=45.91.116.7:4009 , piece_id=HY4OYCNGEFGATDPFK62QZ5OMYM7VIBIGR5DOFXS42D63K2PWHWRQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.464552759 (563h59m55.817337888s) [noise=true , address=45.91.116.6:4975 , piece_id=P2OPDQC5CAUNU6PKCZJ74BHRV4X2ZY7YGZAFSHTWMTLK34ORHHKQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.570763590 (563h59m55.923548719s) [upload=496352]: Next returned nil segment Sep 22 02:08:20.570779229 (563h59m55.923564358s) [upload=496352]: Waiting for error group managing segments... Sep 22 02:08:20.579340004 (563h59m55.932125133s) [noise=true , address=45.91.116.5:4299 , piece_id=7RLKVP75JC53US6SNGULMLNL2VQSBS4C2JXJ2XEJM5FWQN3JKW6A, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.579408563 (563h59m55.932193691s) [noise=true , address=45.91.116.4:4937 , piece_id=TSG3DWXGOAMIKEFOI3GL266ONTV5O5PZVJIJEJVA3GHZVEFLCWBQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.580552390 (563h59m55.933337517s) [noise=true , address=77.90.31.17:4770 , piece_id=LBBHDCEYUGA5IL3SETBBBXN44RR3MPS6CET4WOVHEHBHIRLSOFSA, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.580668457 (563h59m55.933453586s) [noise=true , address=45.91.116.7:4009 , piece_id=HY4OYCNGEFGATDPFK62QZ5OMYM7VIBIGR5DOFXS42D63K2PWHWRQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.581149310 (563h59m55.933934438s) [noise=true , address=45.91.116.6:4975 , piece_id=P2OPDQC5CAUNU6PKCZJ74BHRV4X2ZY7YGZAFSHTWMTLK34ORHHKQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.581266139 (563h59m55.934051267s) [noise=true , address=45.91.116.8:4048 , piece_id=KDUUMIWPL56B4KUYAHZ3SVL4E3EHYGB2NCSJKZESBPYNBCABUR7A, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.581741721 (563h59m55.934526849s) [noise=true , address=77.90.31.14:4663 , piece_id=6IB3E74QRO2AMLSZ3SOL6FHHCIAD2YCLSB327HX7JSSNMJHZ4YNQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.581957146 (563h59m55.934742274s) [noise=true , address=45.91.116.4:4939 , piece_id=ZVT77KYOPEZU5NVKQZNJRLJK6AFBI2IGK77KSGRG4WS26GKPC5AA, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.582328282 (563h59m55.93511341s) [noise=true , address=77.90.31.10:4518 , piece_id=GSZBFIOCXFVRZNCDBC2DFJUJLYHWYW7MUAS7C4XINUX7IH5KUQDQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.582506817 (563h59m55.935291945s) [noise=true , address=45.91.116.9:4262 , piece_id=NM4FUTBS3RJWOHEVN4KMBL4TEEVBIOOSROBAWZWJKLQTWWJMGIBA, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.583549694 (563h59m55.936334832s) [noise=true , address=77.90.31.19:4842 , piece_id=YESLECXUJ5I2NOZXRW6HSET7YNWF6373QNIXCPHWMZHC5TFTYLJA, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:50.400527190 (564h0m25.753312298s) [noise=true , address=145.79.227.15:4876 , piece_id=XPFBENRLP7GSVALZDUMBAIM5Y4S4MS7LGXLNGMNO3RZMLHSFU3GQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: rpc: dial tcp 145.79.227.15:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:50.400557446 (564h0m25.753342545s) [noise=true , address=145.79.227.7:4295 , piece_id=WMEUSUFV3UET3KJIHNKXSJOIBSWFX7YV2F5Z4PN53WG24TSVYKCQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:08:50.400588505 (564h0m25.753373603s) [noise=true , address=145.79.227.11:4731 , piece_id=IOSKWC6JD4IBOLF2IB443K522B67ET7CAG33VGXXQOMJAUFVZT5A, seg_pos={0 0}, upload=496352]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:08:50.400603342 (564h0m25.75338844s) [noise=true , address=145.79.227.16:4909 , piece_id=P3Z2HGUTNOXDJF4TDR4LGXVWKFKUODFMSMHQGWT4NU3C5UYGXBVQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:50.400635312 (564h0m25.753420411s) [noise=true , address=145.79.227.18:4007 , piece_id=X2ILLKTIZSULE6EL2DKZ2V7WZFWCXZ5Z7NMEQQU7ORG6QANYGE5A, seg_pos={0 0}, upload=496352]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: rpc: dial tcp 145.79.227.18:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:50.400663836 (564h0m25.753448944s) [seg_pos={0 0}, upload=496352]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:08:50.427527518 (564h0m25.780312636s) [upload=496352]: Done uploading segments... Sep 22 02:08:50.427543308 (564h0m25.780328426s) [upload=496352]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 431.830298ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................................X.X..] duration: 414.141935ms FAIL (2025-09-22 02:10:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 414.164006ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................................................X.XX] duration: 1m0.019294837s FAIL (2025-09-22 02:10:27 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 9) ======LOGS====== 4h1m2.532856656s) [noise=true , address=145.79.227.11:4763 , piece_id=25OCEOVJWHHW7IL5VCHAPGB53EI5WZVU4QNA3WC4BU2ORO7YPRPQ, seg_pos={0 0}, upload=496367]: Uploading piece... Sep 22 02:09:27.180540258 (564h1m2.533325385s) [noise=true , address=145.79.227.6:4283 , piece_id=VN6INJI2LTDOHEP3MFH2GDO3RXDPAHWOX6WUNDX66QBD4HOMJE6Q, seg_pos={0 0}, upload=496367]: Uploading piece... Sep 22 02:09:27.180554985 (564h1m2.533340113s) [noise=true , address=45.91.116.13:4174 , piece_id=F2D7YJ3WC5KE52POD3U2FYAGOQPJPFIUYFOJC3GAKUTSZRHCK7GQ, seg_pos={0 0}, upload=496367]: Uploading piece... Sep 22 02:09:27.180976246 (564h1m2.533761385s) [noise=true , address=145.79.227.5:4252 , piece_id=6QEJXUQFOUV6TUAQQIZTMVPKGQBULI3BK6KACALY7AONRMGOBZTQ, seg_pos={0 0}, upload=496367]: Uploading piece... Sep 22 02:09:27.180988889 (564h1m2.533774028s) [noise=true , address=77.90.31.10:4526 , piece_id=JAODHJBE3GIJRSHICWH2KSN6OJKS5YS3V22KFJS64XPIFFNFZ4GQ, seg_pos={0 0}, upload=496367]: Uploading piece... Sep 22 02:09:27.181360447 (564h1m2.534145585s) [noise=true , address=145.79.227.7:4322 , piece_id=WRU2N2ZYNQVLRJEQT6CMLZA46HLFDPFLH7ZBGPXQAFLQPPAMOETQ, seg_pos={0 0}, upload=496367]: Uploading piece... Sep 22 02:09:27.181465654 (564h1m2.534250822s) [noise=true , address=145.79.227.39:4016 , piece_id=PL7ASR6MWIHIRH6YB73WCVAXGYGTS5E74U5MUXS3FU2UY2NRRJIQ, seg_pos={0 0}, upload=496367]: Uploading piece... Sep 22 02:09:27.181812073 (564h1m2.534597202s) [noise=true , address=145.79.227.33:4892 , piece_id=RRPIBNJRHRF5O532B5YIZHVPAK4UURCYXNPFFUQCL4HWY354UBWA, seg_pos={0 0}, upload=496367]: Uploading piece... Sep 22 02:09:27.181916811 (564h1m2.534701939s) [noise=true , address=145.79.227.19:4106 , piece_id=F2UJS65NVQHOS5N56YTDQCKVWPOAWXDW4OC76NS634I7R7GAO6FQ, seg_pos={0 0}, upload=496367]: Uploading piece... Sep 22 02:09:27.182172130 (564h1m2.534957258s) [noise=true , address=145.79.227.2:4141 , piece_id=QO32PO2TBMBYVJMOAMLNCI2CDTJTRHG7VV3OMDSOPB5ZMSOKLQPQ, seg_pos={0 0}, upload=496367]: Uploading piece... Sep 22 02:09:27.182278730 (564h1m2.535063858s) [noise=true , address=66.23.193.175:4557 , piece_id=BJDLCI6UWU7L3YMOTWU3BPBH7QRD2IKUPBJIQ4NTE4CAUIC2TMPQ, seg_pos={0 0}, upload=496367]: Uploading piece... Sep 22 02:09:27.182549318 (564h1m2.535334456s) [noise=true , address=145.79.227.18:4018 , piece_id=XTUHZQ7SGVSDJF2CEWBLKSHFRTCN6PTBCXO2EEYEVJDJRW4VFIRQ, seg_pos={0 0}, upload=496367]: Uploading piece... Sep 22 02:09:27.182692105 (564h1m2.535477233s) [noise=true , address=145.79.227.17:4996 , piece_id=ZMFCNOXSIYDMN5KHPDC7VQYQ4DJQR66KKHESJL4H4EVUAMC2VSUQ, seg_pos={0 0}, upload=496367]: Uploading piece... Sep 22 02:09:27.183151137 (564h1m2.535936306s) [noise=true , address=77.90.31.11:4561 , piece_id=YI2VMEPCC3DLJHQ4WPFIIE7X2DPE2U5QMF57M62CJDSA76GRZL3A, seg_pos={0 0}, upload=496367]: Uploading piece... Sep 22 02:09:27.183634073 (564h1m2.536419211s) [noise=true , address=145.79.227.46:4632 , piece_id=XQVDWA2JIV3I7XFEH3QIB2QDBJI6VIS22GKVJRRRVOFZZ6S7URDA, seg_pos={0 0}, upload=496367]: Uploading piece... Sep 22 02:09:27.184692549 (564h1m2.537477677s) [noise=true , address=145.79.227.43:4191 , piece_id=MRZGGOUSWCGCI64VN3URTL2K6NJKYCG27VITOMDXW2JTEQL5U3CA, seg_pos={0 0}, upload=496367]: Uploading piece... Sep 22 02:09:27.185453167 (564h1m2.538238294s) [noise=true , address=145.79.227.14:4861 , piece_id=CHFKY7ZPANSW25MKZQTLAUCVESS6DOPHWYFPGQOKKRH5JFHSHWFQ, seg_pos={0 0}, upload=496367]: Uploading piece... Sep 22 02:09:27.274902185 (564h1m2.627687313s) [upload=496367]: Next returned nil segment Sep 22 02:09:27.274918566 (564h1m2.627703704s) [upload=496367]: Waiting for error group managing segments... Sep 22 02:09:27.284409817 (564h1m2.637194945s) [noise=true , address=77.90.31.11:4561 , piece_id=YI2VMEPCC3DLJHQ4WPFIIE7X2DPE2U5QMF57M62CJDSA76GRZL3A, seg_pos={0 0}, upload=496367]: Done uploading piece. err: <nil> Sep 22 02:09:27.284518621 (564h1m2.637303749s) [noise=true , address=145.79.227.46:4632 , piece_id=XQVDWA2JIV3I7XFEH3QIB2QDBJI6VIS22GKVJRRRVOFZZ6S7URDA, seg_pos={0 0}, upload=496367]: Done uploading piece. err: <nil> Sep 22 02:09:27.285366672 (564h1m2.638151801s) [noise=true , address=145.79.227.26:4663 , piece_id=3Y5DIJ2ED72N64KZ3KHLHBHOTJXVKEMQSKMIUCHVS2L335NHK4EQ, seg_pos={0 0}, upload=496367]: Done uploading piece. err: <nil> Sep 22 02:09:27.286416392 (564h1m2.63920152s) [noise=true , address=145.79.227.33:4892 , piece_id=RRPIBNJRHRF5O532B5YIZHVPAK4UURCYXNPFFUQCL4HWY354UBWA, seg_pos={0 0}, upload=496367]: Done uploading piece. err: <nil> Sep 22 02:09:27.288159673 (564h1m2.640944802s) [noise=true , address=77.90.31.10:4526 , piece_id=JAODHJBE3GIJRSHICWH2KSN6OJKS5YS3V22KFJS64XPIFFNFZ4GQ, seg_pos={0 0}, upload=496367]: Done uploading piece. err: <nil> Sep 22 02:09:27.288643582 (564h1m2.64142871s) [noise=true , address=145.79.227.43:4191 , piece_id=MRZGGOUSWCGCI64VN3URTL2K6NJKYCG27VITOMDXW2JTEQL5U3CA, seg_pos={0 0}, upload=496367]: Done uploading piece. err: <nil> Sep 22 02:09:27.297041562 (564h1m2.649826699s) [noise=true , address=145.79.227.39:4016 , piece_id=PL7ASR6MWIHIRH6YB73WCVAXGYGTS5E74U5MUXS3FU2UY2NRRJIQ, seg_pos={0 0}, upload=496367]: Done uploading piece. err: <nil> Sep 22 02:09:27.305724185 (564h1m2.658509313s) [noise=true , address=45.91.116.13:4174 , piece_id=F2D7YJ3WC5KE52POD3U2FYAGOQPJPFIUYFOJC3GAKUTSZRHCK7GQ, seg_pos={0 0}, upload=496367]: Done uploading piece. err: <nil> Sep 22 02:09:27.799000579 (564h1m3.151785707s) [noise=true , address=66.23.193.175:4557 , piece_id=BJDLCI6UWU7L3YMOTWU3BPBH7QRD2IKUPBJIQ4NTE4CAUIC2TMPQ, seg_pos={0 0}, upload=496367]: Done uploading piece. err: <nil> Sep 22 02:10:27.119428356 (564h2m2.472213484s) [noise=true , address=145.79.227.14:4861 , piece_id=CHFKY7ZPANSW25MKZQTLAUCVESS6DOPHWYFPGQOKKRH5JFHSHWFQ, seg_pos={0 0}, upload=496367]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:145.79.227.14:4861): protocol: rpc: dial tcp 145.79.227.14:4861: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:27.119542000 (564h2m2.472327148s) [noise=true , address=145.79.227.17:4996 , piece_id=ZMFCNOXSIYDMN5KHPDC7VQYQ4DJQR66KKHESJL4H4EVUAMC2VSUQ, seg_pos={0 0}, upload=496367]: Done uploading piece. err: ecclient: upload failed (node:12r6XcE2fb6YxU6kez4UMPeq82jNPi7YiRNmi33tQcTVYMYrMV4, address:145.79.227.17:4996): protocol: rpc: dial tcp 145.79.227.17:4996: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:27.119573007 (564h2m2.472358136s) [noise=true , address=145.79.227.11:4763 , piece_id=25OCEOVJWHHW7IL5VCHAPGB53EI5WZVU4QNA3WC4BU2ORO7YPRPQ, seg_pos={0 0}, upload=496367]: Done uploading piece. err: ecclient: upload failed (node:12P642GmVJzz139WqNE7AZFGieZUk1dNQPwEJ3RSeWVJ4xUZPuZ, address:145.79.227.11:4763): protocol: rpc: dial tcp 145.79.227.11:4763: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:27.119630335 (564h2m2.472415474s) [noise=true , address=145.79.227.18:4018 , piece_id=XTUHZQ7SGVSDJF2CEWBLKSHFRTCN6PTBCXO2EEYEVJDJRW4VFIRQ, seg_pos={0 0}, upload=496367]: Done uploading piece. err: ecclient: upload failed (node:1bFRHbXjTaxoEKc5gbQWYt6jwUg8jknScBZpc8PdKNSZhYqUqP, address:145.79.227.18:4018): protocol: rpc: dial tcp 145.79.227.18:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:27.119650523 (564h2m2.472435651s) [noise=true , address=145.79.227.19:4106 , piece_id=F2UJS65NVQHOS5N56YTDQCKVWPOAWXDW4OC76NS634I7R7GAO6FQ, seg_pos={0 0}, upload=496367]: Done uploading piece. err: ecclient: upload failed (node:12CsQq2BuV9hNc5fVAsw882NkDJMEXZeNwhV2FGNGKAU6wuFBHm, address:145.79.227.19:4106): protocol: rpc: dial tcp 145.79.227.19:4106: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:27.119688374 (564h2m2.472473502s) [noise=true , address=145.79.227.6:4283 , piece_id=VN6INJI2LTDOHEP3MFH2GDO3RXDPAHWOX6WUNDX66QBD4HOMJE6Q, seg_pos={0 0}, upload=496367]: Done uploading piece. err: ecclient: upload failed (node:1ytKjZnmERiZtgJuMJhXG5KFwUQwie27odve4LPE7Pw5hR5ZaU, address:145.79.227.6:4283): protocol: rpc: dial tcp 145.79.227.6:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:27.119712760 (564h2m2.472497888s) [noise=true , address=145.79.227.5:4252 , piece_id=6QEJXUQFOUV6TUAQQIZTMVPKGQBULI3BK6KACALY7AONRMGOBZTQ, seg_pos={0 0}, upload=496367]: Done uploading piece. err: ecclient: upload failed (node:125gP7pSQx5QXXgE7boHtFAtBqaZ19Ti144TEhWdJYwxCzncvCs, address:145.79.227.5:4252): protocol: rpc: dial tcp 145.79.227.5:4252: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:27.119726575 (564h2m2.472511704s) [noise=true , address=145.79.227.2:4141 , piece_id=QO32PO2TBMBYVJMOAMLNCI2CDTJTRHG7VV3OMDSOPB5ZMSOKLQPQ, seg_pos={0 0}, upload=496367]: Done uploading piece. err: ecclient: upload failed (node:129zfpWSguxcUH9723ivdoL39WgpjiTxA189mYbzsGAGVjFcXLL, address:145.79.227.2:4141): protocol: rpc: dial tcp 145.79.227.2:4141: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:27.119747766 (564h2m2.472532895s) [noise=true , address=145.79.227.7:4322 , piece_id=WRU2N2ZYNQVLRJEQT6CMLZA46HLFDPFLH7ZBGPXQAFLQPPAMOETQ, seg_pos={0 0}, upload=496367]: Done uploading piece. err: ecclient: upload failed (node:12WwYqhEYC9QPkGDiSt5bF7izvhpsawWx5dwTqP5zZcPyigs1n3, address:145.79.227.7:4322): protocol: rpc: dial tcp 145.79.227.7:4322: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:27.119786940 (564h2m2.472572068s) [seg_pos={0 0}, upload=496367]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil> Sep 22 02:10:27.136324202 (564h2m2.489109331s) [upload=496367]: Done uploading segments... Sep 22 02:10:27.136345543 (564h2m2.489130681s) [upload=496367]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9) Duration: 1m0.019314604s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................................................XXX...] duration: 410.583557ms FAIL (2025-09-22 02:10:25 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:08:20.420565373 (563h59m55.773350502s) [upload=496352]: Starting upload Sep 22 02:08:20.420650714 (563h59m55.773435842s) [upload=496352]: Done starting upload Sep 22 02:08:20.420786879 (563h59m55.773572007s) [upload=496352]: Uploading segments... Sep 22 02:08:20.421010398 (563h59m55.773795526s) [upload=496352]: Got next segment. Inline: false Sep 22 02:08:20.421056685 (563h59m55.773841813s) [upload=496352]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:08:20.421061874 (563h59m55.773847003s) [upload=496352]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:08:20.460287685 (563h59m55.813072813s) [seg_pos={0 0}, upload=496352]: Begin upload segment... Sep 22 02:08:20.460321128 (563h59m55.813106256s) [seg_pos={0 0}, upload=496352]: Done begin upload segment. Sep 22 02:08:20.460346526 (563h59m55.813131653s) [noise=true , address=45.91.116.5:4299 , piece_id=7RLKVP75JC53US6SNGULMLNL2VQSBS4C2JXJ2XEJM5FWQN3JKW6A, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.460355883 (563h59m55.813141011s) [noise=true , address=145.79.227.7:4295 , piece_id=WMEUSUFV3UET3KJIHNKXSJOIBSWFX7YV2F5Z4PN53WG24TSVYKCQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.460801099 (563h59m55.813586227s) [noise=true , address=145.79.227.11:4731 , piece_id=IOSKWC6JD4IBOLF2IB443K522B67ET7CAG33VGXXQOMJAUFVZT5A, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.460815526 (563h59m55.813600654s) [noise=true , address=77.90.31.14:4663 , piece_id=6IB3E74QRO2AMLSZ3SOL6FHHCIAD2YCLSB327HX7JSSNMJHZ4YNQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461169329 (563h59m55.813954458s) [noise=true , address=77.90.31.10:4518 , piece_id=GSZBFIOCXFVRZNCDBC2DFJUJLYHWYW7MUAS7C4XINUX7IH5KUQDQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461183325 (563h59m55.813968464s) [noise=true , address=145.79.227.16:4909 , piece_id=P3Z2HGUTNOXDJF4TDR4LGXVWKFKUODFMSMHQGWT4NU3C5UYGXBVQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461597703 (563h59m55.814382831s) [noise=true , address=77.90.31.19:4842 , piece_id=YESLECXUJ5I2NOZXRW6HSET7YNWF6373QNIXCPHWMZHC5TFTYLJA, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461608854 (563h59m55.814393982s) [noise=true , address=45.91.116.9:4262 , piece_id=NM4FUTBS3RJWOHEVN4KMBL4TEEVBIOOSROBAWZWJKLQTWWJMGIBA, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461960725 (563h59m55.814745853s) [noise=true , address=45.91.116.4:4937 , piece_id=TSG3DWXGOAMIKEFOI3GL266ONTV5O5PZVJIJEJVA3GHZVEFLCWBQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461986163 (563h59m55.81477129s) [noise=true , address=145.79.227.15:4876 , piece_id=XPFBENRLP7GSVALZDUMBAIM5Y4S4MS7LGXLNGMNO3RZMLHSFU3GQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.462326511 (563h59m55.815111639s) [noise=true , address=77.90.31.17:4770 , piece_id=LBBHDCEYUGA5IL3SETBBBXN44RR3MPS6CET4WOVHEHBHIRLSOFSA, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.462337151 (563h59m55.815122279s) [noise=true , address=45.91.116.8:4048 , piece_id=KDUUMIWPL56B4KUYAHZ3SVL4E3EHYGB2NCSJKZESBPYNBCABUR7A, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.462702948 (563h59m55.815488076s) [noise=true , address=145.79.227.18:4007 , piece_id=X2ILLKTIZSULE6EL2DKZ2V7WZFWCXZ5Z7NMEQQU7ORG6QANYGE5A, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.463054467 (563h59m55.815839596s) [noise=true , address=45.91.116.4:4939 , piece_id=ZVT77KYOPEZU5NVKQZNJRLJK6AFBI2IGK77KSGRG4WS26GKPC5AA, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.463414943 (563h59m55.816200081s) [noise=true , address=45.91.116.7:4009 , piece_id=HY4OYCNGEFGATDPFK62QZ5OMYM7VIBIGR5DOFXS42D63K2PWHWRQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.464552759 (563h59m55.817337888s) [noise=true , address=45.91.116.6:4975 , piece_id=P2OPDQC5CAUNU6PKCZJ74BHRV4X2ZY7YGZAFSHTWMTLK34ORHHKQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.570763590 (563h59m55.923548719s) [upload=496352]: Next returned nil segment Sep 22 02:08:20.570779229 (563h59m55.923564358s) [upload=496352]: Waiting for error group managing segments... Sep 22 02:08:20.579340004 (563h59m55.932125133s) [noise=true , address=45.91.116.5:4299 , piece_id=7RLKVP75JC53US6SNGULMLNL2VQSBS4C2JXJ2XEJM5FWQN3JKW6A, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.579408563 (563h59m55.932193691s) [noise=true , address=45.91.116.4:4937 , piece_id=TSG3DWXGOAMIKEFOI3GL266ONTV5O5PZVJIJEJVA3GHZVEFLCWBQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.580552390 (563h59m55.933337517s) [noise=true , address=77.90.31.17:4770 , piece_id=LBBHDCEYUGA5IL3SETBBBXN44RR3MPS6CET4WOVHEHBHIRLSOFSA, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.580668457 (563h59m55.933453586s) [noise=true , address=45.91.116.7:4009 , piece_id=HY4OYCNGEFGATDPFK62QZ5OMYM7VIBIGR5DOFXS42D63K2PWHWRQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.581149310 (563h59m55.933934438s) [noise=true , address=45.91.116.6:4975 , piece_id=P2OPDQC5CAUNU6PKCZJ74BHRV4X2ZY7YGZAFSHTWMTLK34ORHHKQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.581266139 (563h59m55.934051267s) [noise=true , address=45.91.116.8:4048 , piece_id=KDUUMIWPL56B4KUYAHZ3SVL4E3EHYGB2NCSJKZESBPYNBCABUR7A, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.581741721 (563h59m55.934526849s) [noise=true , address=77.90.31.14:4663 , piece_id=6IB3E74QRO2AMLSZ3SOL6FHHCIAD2YCLSB327HX7JSSNMJHZ4YNQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.581957146 (563h59m55.934742274s) [noise=true , address=45.91.116.4:4939 , piece_id=ZVT77KYOPEZU5NVKQZNJRLJK6AFBI2IGK77KSGRG4WS26GKPC5AA, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.582328282 (563h59m55.93511341s) [noise=true , address=77.90.31.10:4518 , piece_id=GSZBFIOCXFVRZNCDBC2DFJUJLYHWYW7MUAS7C4XINUX7IH5KUQDQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.582506817 (563h59m55.935291945s) [noise=true , address=45.91.116.9:4262 , piece_id=NM4FUTBS3RJWOHEVN4KMBL4TEEVBIOOSROBAWZWJKLQTWWJMGIBA, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.583549694 (563h59m55.936334832s) [noise=true , address=77.90.31.19:4842 , piece_id=YESLECXUJ5I2NOZXRW6HSET7YNWF6373QNIXCPHWMZHC5TFTYLJA, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:50.400527190 (564h0m25.753312298s) [noise=true , address=145.79.227.15:4876 , piece_id=XPFBENRLP7GSVALZDUMBAIM5Y4S4MS7LGXLNGMNO3RZMLHSFU3GQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: rpc: dial tcp 145.79.227.15:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:50.400557446 (564h0m25.753342545s) [noise=true , address=145.79.227.7:4295 , piece_id=WMEUSUFV3UET3KJIHNKXSJOIBSWFX7YV2F5Z4PN53WG24TSVYKCQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:08:50.400588505 (564h0m25.753373603s) [noise=true , address=145.79.227.11:4731 , piece_id=IOSKWC6JD4IBOLF2IB443K522B67ET7CAG33VGXXQOMJAUFVZT5A, seg_pos={0 0}, upload=496352]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:08:50.400603342 (564h0m25.75338844s) [noise=true , address=145.79.227.16:4909 , piece_id=P3Z2HGUTNOXDJF4TDR4LGXVWKFKUODFMSMHQGWT4NU3C5UYGXBVQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:50.400635312 (564h0m25.753420411s) [noise=true , address=145.79.227.18:4007 , piece_id=X2ILLKTIZSULE6EL2DKZ2V7WZFWCXZ5Z7NMEQQU7ORG6QANYGE5A, seg_pos={0 0}, upload=496352]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: rpc: dial tcp 145.79.227.18:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:50.400663836 (564h0m25.753448944s) [seg_pos={0 0}, upload=496352]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:08:50.427527518 (564h0m25.780312636s) [upload=496352]: Done uploading segments... Sep 22 02:08:50.427543308 (564h0m25.780328426s) [upload=496352]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 410.60098ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................................................XXX..] duration: 421.42265ms FAIL (2025-09-22 02:10:07 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:08:20.420565373 (563h59m55.773350502s) [upload=496352]: Starting upload Sep 22 02:08:20.420650714 (563h59m55.773435842s) [upload=496352]: Done starting upload Sep 22 02:08:20.420786879 (563h59m55.773572007s) [upload=496352]: Uploading segments... Sep 22 02:08:20.421010398 (563h59m55.773795526s) [upload=496352]: Got next segment. Inline: false Sep 22 02:08:20.421056685 (563h59m55.773841813s) [upload=496352]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:08:20.421061874 (563h59m55.773847003s) [upload=496352]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:08:20.460287685 (563h59m55.813072813s) [seg_pos={0 0}, upload=496352]: Begin upload segment... Sep 22 02:08:20.460321128 (563h59m55.813106256s) [seg_pos={0 0}, upload=496352]: Done begin upload segment. Sep 22 02:08:20.460346526 (563h59m55.813131653s) [noise=true , address=45.91.116.5:4299 , piece_id=7RLKVP75JC53US6SNGULMLNL2VQSBS4C2JXJ2XEJM5FWQN3JKW6A, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.460355883 (563h59m55.813141011s) [noise=true , address=145.79.227.7:4295 , piece_id=WMEUSUFV3UET3KJIHNKXSJOIBSWFX7YV2F5Z4PN53WG24TSVYKCQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.460801099 (563h59m55.813586227s) [noise=true , address=145.79.227.11:4731 , piece_id=IOSKWC6JD4IBOLF2IB443K522B67ET7CAG33VGXXQOMJAUFVZT5A, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.460815526 (563h59m55.813600654s) [noise=true , address=77.90.31.14:4663 , piece_id=6IB3E74QRO2AMLSZ3SOL6FHHCIAD2YCLSB327HX7JSSNMJHZ4YNQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461169329 (563h59m55.813954458s) [noise=true , address=77.90.31.10:4518 , piece_id=GSZBFIOCXFVRZNCDBC2DFJUJLYHWYW7MUAS7C4XINUX7IH5KUQDQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461183325 (563h59m55.813968464s) [noise=true , address=145.79.227.16:4909 , piece_id=P3Z2HGUTNOXDJF4TDR4LGXVWKFKUODFMSMHQGWT4NU3C5UYGXBVQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461597703 (563h59m55.814382831s) [noise=true , address=77.90.31.19:4842 , piece_id=YESLECXUJ5I2NOZXRW6HSET7YNWF6373QNIXCPHWMZHC5TFTYLJA, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461608854 (563h59m55.814393982s) [noise=true , address=45.91.116.9:4262 , piece_id=NM4FUTBS3RJWOHEVN4KMBL4TEEVBIOOSROBAWZWJKLQTWWJMGIBA, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461960725 (563h59m55.814745853s) [noise=true , address=45.91.116.4:4937 , piece_id=TSG3DWXGOAMIKEFOI3GL266ONTV5O5PZVJIJEJVA3GHZVEFLCWBQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461986163 (563h59m55.81477129s) [noise=true , address=145.79.227.15:4876 , piece_id=XPFBENRLP7GSVALZDUMBAIM5Y4S4MS7LGXLNGMNO3RZMLHSFU3GQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.462326511 (563h59m55.815111639s) [noise=true , address=77.90.31.17:4770 , piece_id=LBBHDCEYUGA5IL3SETBBBXN44RR3MPS6CET4WOVHEHBHIRLSOFSA, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.462337151 (563h59m55.815122279s) [noise=true , address=45.91.116.8:4048 , piece_id=KDUUMIWPL56B4KUYAHZ3SVL4E3EHYGB2NCSJKZESBPYNBCABUR7A, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.462702948 (563h59m55.815488076s) [noise=true , address=145.79.227.18:4007 , piece_id=X2ILLKTIZSULE6EL2DKZ2V7WZFWCXZ5Z7NMEQQU7ORG6QANYGE5A, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.463054467 (563h59m55.815839596s) [noise=true , address=45.91.116.4:4939 , piece_id=ZVT77KYOPEZU5NVKQZNJRLJK6AFBI2IGK77KSGRG4WS26GKPC5AA, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.463414943 (563h59m55.816200081s) [noise=true , address=45.91.116.7:4009 , piece_id=HY4OYCNGEFGATDPFK62QZ5OMYM7VIBIGR5DOFXS42D63K2PWHWRQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.464552759 (563h59m55.817337888s) [noise=true , address=45.91.116.6:4975 , piece_id=P2OPDQC5CAUNU6PKCZJ74BHRV4X2ZY7YGZAFSHTWMTLK34ORHHKQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.570763590 (563h59m55.923548719s) [upload=496352]: Next returned nil segment Sep 22 02:08:20.570779229 (563h59m55.923564358s) [upload=496352]: Waiting for error group managing segments... Sep 22 02:08:20.579340004 (563h59m55.932125133s) [noise=true , address=45.91.116.5:4299 , piece_id=7RLKVP75JC53US6SNGULMLNL2VQSBS4C2JXJ2XEJM5FWQN3JKW6A, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.579408563 (563h59m55.932193691s) [noise=true , address=45.91.116.4:4937 , piece_id=TSG3DWXGOAMIKEFOI3GL266ONTV5O5PZVJIJEJVA3GHZVEFLCWBQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.580552390 (563h59m55.933337517s) [noise=true , address=77.90.31.17:4770 , piece_id=LBBHDCEYUGA5IL3SETBBBXN44RR3MPS6CET4WOVHEHBHIRLSOFSA, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.580668457 (563h59m55.933453586s) [noise=true , address=45.91.116.7:4009 , piece_id=HY4OYCNGEFGATDPFK62QZ5OMYM7VIBIGR5DOFXS42D63K2PWHWRQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.581149310 (563h59m55.933934438s) [noise=true , address=45.91.116.6:4975 , piece_id=P2OPDQC5CAUNU6PKCZJ74BHRV4X2ZY7YGZAFSHTWMTLK34ORHHKQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.581266139 (563h59m55.934051267s) [noise=true , address=45.91.116.8:4048 , piece_id=KDUUMIWPL56B4KUYAHZ3SVL4E3EHYGB2NCSJKZESBPYNBCABUR7A, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.581741721 (563h59m55.934526849s) [noise=true , address=77.90.31.14:4663 , piece_id=6IB3E74QRO2AMLSZ3SOL6FHHCIAD2YCLSB327HX7JSSNMJHZ4YNQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.581957146 (563h59m55.934742274s) [noise=true , address=45.91.116.4:4939 , piece_id=ZVT77KYOPEZU5NVKQZNJRLJK6AFBI2IGK77KSGRG4WS26GKPC5AA, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.582328282 (563h59m55.93511341s) [noise=true , address=77.90.31.10:4518 , piece_id=GSZBFIOCXFVRZNCDBC2DFJUJLYHWYW7MUAS7C4XINUX7IH5KUQDQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.582506817 (563h59m55.935291945s) [noise=true , address=45.91.116.9:4262 , piece_id=NM4FUTBS3RJWOHEVN4KMBL4TEEVBIOOSROBAWZWJKLQTWWJMGIBA, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.583549694 (563h59m55.936334832s) [noise=true , address=77.90.31.19:4842 , piece_id=YESLECXUJ5I2NOZXRW6HSET7YNWF6373QNIXCPHWMZHC5TFTYLJA, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:50.400527190 (564h0m25.753312298s) [noise=true , address=145.79.227.15:4876 , piece_id=XPFBENRLP7GSVALZDUMBAIM5Y4S4MS7LGXLNGMNO3RZMLHSFU3GQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: rpc: dial tcp 145.79.227.15:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:50.400557446 (564h0m25.753342545s) [noise=true , address=145.79.227.7:4295 , piece_id=WMEUSUFV3UET3KJIHNKXSJOIBSWFX7YV2F5Z4PN53WG24TSVYKCQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:08:50.400588505 (564h0m25.753373603s) [noise=true , address=145.79.227.11:4731 , piece_id=IOSKWC6JD4IBOLF2IB443K522B67ET7CAG33VGXXQOMJAUFVZT5A, seg_pos={0 0}, upload=496352]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:08:50.400603342 (564h0m25.75338844s) [noise=true , address=145.79.227.16:4909 , piece_id=P3Z2HGUTNOXDJF4TDR4LGXVWKFKUODFMSMHQGWT4NU3C5UYGXBVQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:50.400635312 (564h0m25.753420411s) [noise=true , address=145.79.227.18:4007 , piece_id=X2ILLKTIZSULE6EL2DKZ2V7WZFWCXZ5Z7NMEQQU7ORG6QANYGE5A, seg_pos={0 0}, upload=496352]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: rpc: dial tcp 145.79.227.18:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:50.400663836 (564h0m25.753448944s) [seg_pos={0 0}, upload=496352]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:08:50.427527518 (564h0m25.780312636s) [upload=496352]: Done uploading segments... Sep 22 02:08:50.427543308 (564h0m25.780328426s) [upload=496352]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 421.448389ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................................X.X.] duration: 405.980268ms FAIL (2025-09-22 02:10:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 406.046914ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................X.X] duration: 30.000371216s FAIL (2025-09-22 02:09:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000416841s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................X.X] duration: 1m0.036828937s FAIL (2025-09-22 02:09:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 02:08:26.167522827 (564h0m1.520307935s) [upload=496354]: Starting upload Sep 22 02:08:26.167598459 (564h0m1.520383567s) [upload=496354]: Done starting upload Sep 22 02:08:26.167751216 (564h0m1.520536314s) [upload=496354]: Uploading segments... Sep 22 02:08:26.167840573 (564h0m1.520625701s) [upload=496354]: Got next segment. Inline: false Sep 22 02:08:26.167900465 (564h0m1.520685564s) [upload=496354]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:08:26.167906026 (564h0m1.520691124s) [upload=496354]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:08:26.222580746 (564h0m1.575365854s) [seg_pos={0 0}, upload=496354]: Begin upload segment... Sep 22 02:08:26.222608989 (564h0m1.575394086s) [seg_pos={0 0}, upload=496354]: Done begin upload segment. Sep 22 02:08:26.222629588 (564h0m1.575414685s) [noise=true , address=145.79.227.25:4650 , piece_id=3YEPH3BXAQ3STFSJXA7EBLS3BVDSFYHXTTFEVAXRWQC7DZOGM3OQ, seg_pos={0 0}, upload=496354]: Uploading piece... Sep 22 02:08:26.222650427 (564h0m1.575435535s) [noise=true , address=145.79.227.23:4575 , piece_id=XCEZ4ZXZM7HMQD47VQIBWXXZBAYSCNJW3ZEGS7H2JGM4H6CIY5CQ, seg_pos={0 0}, upload=496354]: Uploading piece... Sep 22 02:08:26.222907390 (564h0m1.575692487s) [noise=true , address=145.79.227.2:4136 , piece_id=G3V6MLAHEGROIGKTFWFTDF3EDAFDMIBTJOZ2G3LNOCZSJA2QZR4Q, seg_pos={0 0}, upload=496354]: Uploading piece... Sep 22 02:08:26.223013488 (564h0m1.575798596s) [noise=true , address=145.79.227.17:4993 , piece_id=J46GLHBERV7GVGNFOKBJK3EVQ7DOJVHGCZ5AU5JEELHU55BB2PDQ, seg_pos={0 0}, upload=496354]: Uploading piece... Sep 22 02:08:26.223118666 (564h0m1.575903774s) [noise=true , address=145.79.227.47:4291 , piece_id=YFGHX4VQ2OJOPRT2GP6X5IPHE3NAWZDHFVT5WWAVXUDJCDAWFDXQ, seg_pos={0 0}, upload=496354]: Uploading piece... Sep 22 02:08:26.223292812 (564h0m1.57607792s) [noise=true , address=77.90.31.11:4586 , piece_id=Z3HAZV3YQMXMFNGGIJGDZNQCIVG2RI26CR7ASKIP6LJPTSJU42KA, seg_pos={0 0}, upload=496354]: Uploading piece... Sep 22 02:08:26.223334901 (564h0m1.576120009s) [noise=true , address=145.79.227.5:4237 , piece_id=DNJRZMCJV5GBNFJA3DICQE3VNZJS6JDT5WKIM4NNJIBBCHUJ7TPQ, seg_pos={0 0}, upload=496354]: Uploading piece... Sep 22 02:08:26.223571325 (564h0m1.576356433s) [noise=true , address=145.79.227.24:4591 , piece_id=PNZAPG2TL4RSSWWCVMC7XN3CHYF6TGZMGOQVTQQDUIHY62HIM2UA, seg_pos={0 0}, upload=496354]: Uploading piece... Sep 22 02:08:26.223596211 (564h0m1.57638131s) [noise=true , address=145.79.227.27:4700 , piece_id=WRR7HG4VDIAFT7VULCQBTMFZ7J2FXOMDRNDRLBEMVGXRQO6F6SGA, seg_pos={0 0}, upload=496354]: Uploading piece... Sep 22 02:08:26.223775087 (564h0m1.576560186s) [noise=true , address=145.79.227.26:4678 , piece_id=G34Z434GV34SDAQQ44QZB5IVEV7QLBZKDFIJ5RMJ5R3CVJGLG6TA, seg_pos={0 0}, upload=496354]: Uploading piece... Sep 22 02:08:26.223864816 (564h0m1.576649924s) [noise=true , address=145.79.227.41:4100 , piece_id=6IDZFVCKRXON6UVMIYFYQ2FEKCA325BOQOWP65LRI6ZJVDWR3KUA, seg_pos={0 0}, upload=496354]: Uploading piece... Sep 22 02:08:26.223998206 (564h0m1.576783314s) [noise=true , address=145.79.227.46:4629 , piece_id=PHWS6U6XHIRIJN7V3NUVEUMA74OY2XPGWNSZSUBYY7FSKEHLSAWA, seg_pos={0 0}, upload=496354]: Uploading piece... Sep 22 02:08:26.224153237 (564h0m1.576938335s) [noise=true , address=145.79.227.52:4474 , piece_id=K5MWP5UOTMSC2H37D3MYUHDJRXVBY75ZZ4HBSOFLF2LZHX7IIEHA, seg_pos={0 0}, upload=496354]: Uploading piece... Sep 22 02:08:26.224206788 (564h0m1.576991896s) [noise=true , address=45.91.116.13:4173 , piece_id=JTI2J6I3MQXPLPQUOWU4UJC65FBLTI4NPFDU7ZFYP4EDOJNSZO7A, seg_pos={0 0}, upload=496354]: Uploading piece... Sep 22 02:08:26.224409007 (564h0m1.577194115s) [noise=true , address=145.79.227.7:4299 , piece_id=OQO6MCT5YXMOIS236UQ5JFK4EKMZO3CBZDNZHSXLNJSTEYOVVEWA, seg_pos={0 0}, upload=496354]: Uploading piece... Sep 22 02:08:26.224447499 (564h0m1.577232607s) [noise=true , address=145.79.227.28:4737 , piece_id=7PZZC7CAJ5ZRTNVH7TZVGZ5QECIBH6M6ZAZFDQXK2PODOO2PJ42A, seg_pos={0 0}, upload=496354]: Uploading piece... Sep 22 02:08:26.224695244 (564h0m1.577480343s) [noise=true , address=145.79.227.6:4284 , piece_id=UTV3DG67OZSBUQBB4CNBWZ3SSOCSHJE5PXIWVGA7ZYCTPD5P4XWA, seg_pos={0 0}, upload=496354]: Uploading piece... Sep 22 02:08:26.224824727 (564h0m1.577609825s) [noise=true , address=145.79.227.22:4518 , piece_id=6KYEL2QKXWJK47TUTWJRXHIQQIPSCHVFPR7AJEWBPTC36FQSESJA, seg_pos={0 0}, upload=496354]: Uploading piece... Sep 22 02:08:26.295534820 (564h0m1.648319918s) [upload=496354]: Next returned nil segment Sep 22 02:08:26.295547393 (564h0m1.648332492s) [upload=496354]: Waiting for error group managing segments... Sep 22 02:08:26.305381027 (564h0m1.658166135s) [noise=true , address=145.79.227.28:4737 , piece_id=7PZZC7CAJ5ZRTNVH7TZVGZ5QECIBH6M6ZAZFDQXK2PODOO2PJ42A, seg_pos={0 0}, upload=496354]: Done uploading piece. err: <nil> Sep 22 02:08:26.305746403 (564h0m1.658531511s) [noise=true , address=145.79.227.25:4650 , piece_id=3YEPH3BXAQ3STFSJXA7EBLS3BVDSFYHXTTFEVAXRWQC7DZOGM3OQ, seg_pos={0 0}, upload=496354]: Done uploading piece. err: <nil> Sep 22 02:08:26.306652974 (564h0m1.659438082s) [noise=true , address=77.90.31.11:4586 , piece_id=Z3HAZV3YQMXMFNGGIJGDZNQCIVG2RI26CR7ASKIP6LJPTSJU42KA, seg_pos={0 0}, upload=496354]: Done uploading piece. err: <nil> Sep 22 02:08:26.313193167 (564h0m1.665978275s) [noise=true , address=145.79.227.47:4291 , piece_id=YFGHX4VQ2OJOPRT2GP6X5IPHE3NAWZDHFVT5WWAVXUDJCDAWFDXQ, seg_pos={0 0}, upload=496354]: Done uploading piece. err: <nil> Sep 22 02:08:26.320312658 (564h0m1.673097776s) [noise=true , address=145.79.227.52:4474 , piece_id=K5MWP5UOTMSC2H37D3MYUHDJRXVBY75ZZ4HBSOFLF2LZHX7IIEHA, seg_pos={0 0}, upload=496354]: Done uploading piece. err: <nil> Sep 22 02:08:26.322046921 (564h0m1.674832029s) [noise=true , address=45.91.116.13:4173 , piece_id=JTI2J6I3MQXPLPQUOWU4UJC65FBLTI4NPFDU7ZFYP4EDOJNSZO7A, seg_pos={0 0}, upload=496354]: Done uploading piece. err: <nil> Sep 22 02:08:26.329563257 (564h0m1.682348394s) [noise=true , address=145.79.227.46:4629 , piece_id=PHWS6U6XHIRIJN7V3NUVEUMA74OY2XPGWNSZSUBYY7FSKEHLSAWA, seg_pos={0 0}, upload=496354]: Done uploading piece. err: <nil> Sep 22 02:08:26.329828774 (564h0m1.682613882s) [noise=true , address=145.79.227.27:4700 , piece_id=WRR7HG4VDIAFT7VULCQBTMFZ7J2FXOMDRNDRLBEMVGXRQO6F6SGA, seg_pos={0 0}, upload=496354]: Done uploading piece. err: <nil> Sep 22 02:08:26.330844550 (564h0m1.683629678s) [noise=true , address=145.79.227.23:4575 , piece_id=XCEZ4ZXZM7HMQD47VQIBWXXZBAYSCNJW3ZEGS7H2JGM4H6CIY5CQ, seg_pos={0 0}, upload=496354]: Done uploading piece. err: <nil> Sep 22 02:08:26.338156943 (564h0m1.690942051s) [noise=true , address=145.79.227.41:4100 , piece_id=6IDZFVCKRXON6UVMIYFYQ2FEKCA325BOQOWP65LRI6ZJVDWR3KUA, seg_pos={0 0}, upload=496354]: Done uploading piece. err: <nil> Sep 22 02:08:26.346966035 (564h0m1.699751162s) [noise=true , address=145.79.227.24:4591 , piece_id=PNZAPG2TL4RSSWWCVMC7XN3CHYF6TGZMGOQVTQQDUIHY62HIM2UA, seg_pos={0 0}, upload=496354]: Done uploading piece. err: <nil> Sep 22 02:08:26.351204117 (564h0m1.703989225s) [noise=true , address=145.79.227.26:4678 , piece_id=G34Z434GV34SDAQQ44QZB5IVEV7QLBZKDFIJ5RMJ5R3CVJGLG6TA, seg_pos={0 0}, upload=496354]: Done uploading piece. err: <nil> Sep 22 02:08:26.352296076 (564h0m1.705081214s) [noise=true , address=145.79.227.22:4518 , piece_id=6KYEL2QKXWJK47TUTWJRXHIQQIPSCHVFPR7AJEWBPTC36FQSESJA, seg_pos={0 0}, upload=496354]: Done uploading piece. err: <nil> Sep 22 02:09:26.153344565 (564h1m1.506129663s) [noise=true , address=145.79.227.7:4299 , piece_id=OQO6MCT5YXMOIS236UQ5JFK4EKMZO3CBZDNZHSXLNJSTEYOVVEWA, seg_pos={0 0}, upload=496354]: Done uploading piece. err: ecclient: upload failed (node:12sX7515Lspa6uoZAfWzqPhmnMPcvrVaASSPungFuZEBhmApN9n, address:145.79.227.7:4299): protocol: rpc: dial tcp 145.79.227.7:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:09:26.153386855 (564h1m1.506171963s) [noise=true , address=145.79.227.17:4993 , piece_id=J46GLHBERV7GVGNFOKBJK3EVQ7DOJVHGCZ5AU5JEELHU55BB2PDQ, seg_pos={0 0}, upload=496354]: Done uploading piece. err: ecclient: upload failed (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD, address:145.79.227.17:4993): protocol: rpc: dial tcp 145.79.227.17:4993: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:09:26.153453430 (564h1m1.506238528s) [noise=true , address=145.79.227.6:4284 , piece_id=UTV3DG67OZSBUQBB4CNBWZ3SSOCSHJE5PXIWVGA7ZYCTPD5P4XWA, seg_pos={0 0}, upload=496354]: Done uploading piece. err: ecclient: upload failed (node:1YAnLu8onh19Mx524VikQzCHEFW4U8dgtuLx8amtQAiReSCisF, address:145.79.227.6:4284): protocol: rpc: dial tcp 145.79.227.6:4284: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:09:26.153558076 (564h1m1.506343184s) [noise=true , address=145.79.227.2:4136 , piece_id=G3V6MLAHEGROIGKTFWFTDF3EDAFDMIBTJOZ2G3LNOCZSJA2QZR4Q, seg_pos={0 0}, upload=496354]: Done uploading piece. err: ecclient: upload failed (node:127n8CMbZQ7haiv6tGHrVg9V8Yf12sYkcbL9mEbSpQ4WkT9isnp, address:145.79.227.2:4136): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:09:26.153566562 (564h1m1.50635167s) [noise=true , address=145.79.227.5:4237 , piece_id=DNJRZMCJV5GBNFJA3DICQE3VNZJS6JDT5WKIM4NNJIBBCHUJ7TPQ, seg_pos={0 0}, upload=496354]: Done uploading piece. err: ecclient: upload failed (node:1o2jLTwKNSjxjFEmckxkG1c5EuySK3o3sawGEikd2Ttz4CWfLa, address:145.79.227.5:4237): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:09:26.153604292 (564h1m1.506389391s) [seg_pos={0 0}, upload=496354]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 02:09:26.188732474 (564h1m1.541517582s) [upload=496354]: Done uploading segments... Sep 22 02:09:26.188751128 (564h1m1.541536227s) [upload=496354]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.036849375s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................................................XXX.] duration: 504.900358ms FAIL (2025-09-22 02:09:03 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:08:20.420565373 (563h59m55.773350502s) [upload=496352]: Starting upload Sep 22 02:08:20.420650714 (563h59m55.773435842s) [upload=496352]: Done starting upload Sep 22 02:08:20.420786879 (563h59m55.773572007s) [upload=496352]: Uploading segments... Sep 22 02:08:20.421010398 (563h59m55.773795526s) [upload=496352]: Got next segment. Inline: false Sep 22 02:08:20.421056685 (563h59m55.773841813s) [upload=496352]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:08:20.421061874 (563h59m55.773847003s) [upload=496352]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:08:20.460287685 (563h59m55.813072813s) [seg_pos={0 0}, upload=496352]: Begin upload segment... Sep 22 02:08:20.460321128 (563h59m55.813106256s) [seg_pos={0 0}, upload=496352]: Done begin upload segment. Sep 22 02:08:20.460346526 (563h59m55.813131653s) [noise=true , address=45.91.116.5:4299 , piece_id=7RLKVP75JC53US6SNGULMLNL2VQSBS4C2JXJ2XEJM5FWQN3JKW6A, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.460355883 (563h59m55.813141011s) [noise=true , address=145.79.227.7:4295 , piece_id=WMEUSUFV3UET3KJIHNKXSJOIBSWFX7YV2F5Z4PN53WG24TSVYKCQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.460801099 (563h59m55.813586227s) [noise=true , address=145.79.227.11:4731 , piece_id=IOSKWC6JD4IBOLF2IB443K522B67ET7CAG33VGXXQOMJAUFVZT5A, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.460815526 (563h59m55.813600654s) [noise=true , address=77.90.31.14:4663 , piece_id=6IB3E74QRO2AMLSZ3SOL6FHHCIAD2YCLSB327HX7JSSNMJHZ4YNQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461169329 (563h59m55.813954458s) [noise=true , address=77.90.31.10:4518 , piece_id=GSZBFIOCXFVRZNCDBC2DFJUJLYHWYW7MUAS7C4XINUX7IH5KUQDQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461183325 (563h59m55.813968464s) [noise=true , address=145.79.227.16:4909 , piece_id=P3Z2HGUTNOXDJF4TDR4LGXVWKFKUODFMSMHQGWT4NU3C5UYGXBVQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461597703 (563h59m55.814382831s) [noise=true , address=77.90.31.19:4842 , piece_id=YESLECXUJ5I2NOZXRW6HSET7YNWF6373QNIXCPHWMZHC5TFTYLJA, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461608854 (563h59m55.814393982s) [noise=true , address=45.91.116.9:4262 , piece_id=NM4FUTBS3RJWOHEVN4KMBL4TEEVBIOOSROBAWZWJKLQTWWJMGIBA, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461960725 (563h59m55.814745853s) [noise=true , address=45.91.116.4:4937 , piece_id=TSG3DWXGOAMIKEFOI3GL266ONTV5O5PZVJIJEJVA3GHZVEFLCWBQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461986163 (563h59m55.81477129s) [noise=true , address=145.79.227.15:4876 , piece_id=XPFBENRLP7GSVALZDUMBAIM5Y4S4MS7LGXLNGMNO3RZMLHSFU3GQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.462326511 (563h59m55.815111639s) [noise=true , address=77.90.31.17:4770 , piece_id=LBBHDCEYUGA5IL3SETBBBXN44RR3MPS6CET4WOVHEHBHIRLSOFSA, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.462337151 (563h59m55.815122279s) [noise=true , address=45.91.116.8:4048 , piece_id=KDUUMIWPL56B4KUYAHZ3SVL4E3EHYGB2NCSJKZESBPYNBCABUR7A, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.462702948 (563h59m55.815488076s) [noise=true , address=145.79.227.18:4007 , piece_id=X2ILLKTIZSULE6EL2DKZ2V7WZFWCXZ5Z7NMEQQU7ORG6QANYGE5A, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.463054467 (563h59m55.815839596s) [noise=true , address=45.91.116.4:4939 , piece_id=ZVT77KYOPEZU5NVKQZNJRLJK6AFBI2IGK77KSGRG4WS26GKPC5AA, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.463414943 (563h59m55.816200081s) [noise=true , address=45.91.116.7:4009 , piece_id=HY4OYCNGEFGATDPFK62QZ5OMYM7VIBIGR5DOFXS42D63K2PWHWRQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.464552759 (563h59m55.817337888s) [noise=true , address=45.91.116.6:4975 , piece_id=P2OPDQC5CAUNU6PKCZJ74BHRV4X2ZY7YGZAFSHTWMTLK34ORHHKQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.570763590 (563h59m55.923548719s) [upload=496352]: Next returned nil segment Sep 22 02:08:20.570779229 (563h59m55.923564358s) [upload=496352]: Waiting for error group managing segments... Sep 22 02:08:20.579340004 (563h59m55.932125133s) [noise=true , address=45.91.116.5:4299 , piece_id=7RLKVP75JC53US6SNGULMLNL2VQSBS4C2JXJ2XEJM5FWQN3JKW6A, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.579408563 (563h59m55.932193691s) [noise=true , address=45.91.116.4:4937 , piece_id=TSG3DWXGOAMIKEFOI3GL266ONTV5O5PZVJIJEJVA3GHZVEFLCWBQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.580552390 (563h59m55.933337517s) [noise=true , address=77.90.31.17:4770 , piece_id=LBBHDCEYUGA5IL3SETBBBXN44RR3MPS6CET4WOVHEHBHIRLSOFSA, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.580668457 (563h59m55.933453586s) [noise=true , address=45.91.116.7:4009 , piece_id=HY4OYCNGEFGATDPFK62QZ5OMYM7VIBIGR5DOFXS42D63K2PWHWRQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.581149310 (563h59m55.933934438s) [noise=true , address=45.91.116.6:4975 , piece_id=P2OPDQC5CAUNU6PKCZJ74BHRV4X2ZY7YGZAFSHTWMTLK34ORHHKQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.581266139 (563h59m55.934051267s) [noise=true , address=45.91.116.8:4048 , piece_id=KDUUMIWPL56B4KUYAHZ3SVL4E3EHYGB2NCSJKZESBPYNBCABUR7A, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.581741721 (563h59m55.934526849s) [noise=true , address=77.90.31.14:4663 , piece_id=6IB3E74QRO2AMLSZ3SOL6FHHCIAD2YCLSB327HX7JSSNMJHZ4YNQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.581957146 (563h59m55.934742274s) [noise=true , address=45.91.116.4:4939 , piece_id=ZVT77KYOPEZU5NVKQZNJRLJK6AFBI2IGK77KSGRG4WS26GKPC5AA, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.582328282 (563h59m55.93511341s) [noise=true , address=77.90.31.10:4518 , piece_id=GSZBFIOCXFVRZNCDBC2DFJUJLYHWYW7MUAS7C4XINUX7IH5KUQDQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.582506817 (563h59m55.935291945s) [noise=true , address=45.91.116.9:4262 , piece_id=NM4FUTBS3RJWOHEVN4KMBL4TEEVBIOOSROBAWZWJKLQTWWJMGIBA, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.583549694 (563h59m55.936334832s) [noise=true , address=77.90.31.19:4842 , piece_id=YESLECXUJ5I2NOZXRW6HSET7YNWF6373QNIXCPHWMZHC5TFTYLJA, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:50.400527190 (564h0m25.753312298s) [noise=true , address=145.79.227.15:4876 , piece_id=XPFBENRLP7GSVALZDUMBAIM5Y4S4MS7LGXLNGMNO3RZMLHSFU3GQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: rpc: dial tcp 145.79.227.15:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:50.400557446 (564h0m25.753342545s) [noise=true , address=145.79.227.7:4295 , piece_id=WMEUSUFV3UET3KJIHNKXSJOIBSWFX7YV2F5Z4PN53WG24TSVYKCQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:08:50.400588505 (564h0m25.753373603s) [noise=true , address=145.79.227.11:4731 , piece_id=IOSKWC6JD4IBOLF2IB443K522B67ET7CAG33VGXXQOMJAUFVZT5A, seg_pos={0 0}, upload=496352]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:08:50.400603342 (564h0m25.75338844s) [noise=true , address=145.79.227.16:4909 , piece_id=P3Z2HGUTNOXDJF4TDR4LGXVWKFKUODFMSMHQGWT4NU3C5UYGXBVQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:50.400635312 (564h0m25.753420411s) [noise=true , address=145.79.227.18:4007 , piece_id=X2ILLKTIZSULE6EL2DKZ2V7WZFWCXZ5Z7NMEQQU7ORG6QANYGE5A, seg_pos={0 0}, upload=496352]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: rpc: dial tcp 145.79.227.18:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:50.400663836 (564h0m25.753448944s) [seg_pos={0 0}, upload=496352]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:08:50.427527518 (564h0m25.780312636s) [upload=496352]: Done uploading segments... Sep 22 02:08:50.427543308 (564h0m25.780328426s) [upload=496352]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 504.925436ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 30.028295068s FAIL (2025-09-22 02:08:50 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:08:20.420565373 (563h59m55.773350502s) [upload=496352]: Starting upload Sep 22 02:08:20.420650714 (563h59m55.773435842s) [upload=496352]: Done starting upload Sep 22 02:08:20.420786879 (563h59m55.773572007s) [upload=496352]: Uploading segments... Sep 22 02:08:20.421010398 (563h59m55.773795526s) [upload=496352]: Got next segment. Inline: false Sep 22 02:08:20.421056685 (563h59m55.773841813s) [upload=496352]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:08:20.421061874 (563h59m55.773847003s) [upload=496352]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:08:20.460287685 (563h59m55.813072813s) [seg_pos={0 0}, upload=496352]: Begin upload segment... Sep 22 02:08:20.460321128 (563h59m55.813106256s) [seg_pos={0 0}, upload=496352]: Done begin upload segment. Sep 22 02:08:20.460346526 (563h59m55.813131653s) [noise=true , address=45.91.116.5:4299 , piece_id=7RLKVP75JC53US6SNGULMLNL2VQSBS4C2JXJ2XEJM5FWQN3JKW6A, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.460355883 (563h59m55.813141011s) [noise=true , address=145.79.227.7:4295 , piece_id=WMEUSUFV3UET3KJIHNKXSJOIBSWFX7YV2F5Z4PN53WG24TSVYKCQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.460801099 (563h59m55.813586227s) [noise=true , address=145.79.227.11:4731 , piece_id=IOSKWC6JD4IBOLF2IB443K522B67ET7CAG33VGXXQOMJAUFVZT5A, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.460815526 (563h59m55.813600654s) [noise=true , address=77.90.31.14:4663 , piece_id=6IB3E74QRO2AMLSZ3SOL6FHHCIAD2YCLSB327HX7JSSNMJHZ4YNQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461169329 (563h59m55.813954458s) [noise=true , address=77.90.31.10:4518 , piece_id=GSZBFIOCXFVRZNCDBC2DFJUJLYHWYW7MUAS7C4XINUX7IH5KUQDQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461183325 (563h59m55.813968464s) [noise=true , address=145.79.227.16:4909 , piece_id=P3Z2HGUTNOXDJF4TDR4LGXVWKFKUODFMSMHQGWT4NU3C5UYGXBVQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461597703 (563h59m55.814382831s) [noise=true , address=77.90.31.19:4842 , piece_id=YESLECXUJ5I2NOZXRW6HSET7YNWF6373QNIXCPHWMZHC5TFTYLJA, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461608854 (563h59m55.814393982s) [noise=true , address=45.91.116.9:4262 , piece_id=NM4FUTBS3RJWOHEVN4KMBL4TEEVBIOOSROBAWZWJKLQTWWJMGIBA, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461960725 (563h59m55.814745853s) [noise=true , address=45.91.116.4:4937 , piece_id=TSG3DWXGOAMIKEFOI3GL266ONTV5O5PZVJIJEJVA3GHZVEFLCWBQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.461986163 (563h59m55.81477129s) [noise=true , address=145.79.227.15:4876 , piece_id=XPFBENRLP7GSVALZDUMBAIM5Y4S4MS7LGXLNGMNO3RZMLHSFU3GQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.462326511 (563h59m55.815111639s) [noise=true , address=77.90.31.17:4770 , piece_id=LBBHDCEYUGA5IL3SETBBBXN44RR3MPS6CET4WOVHEHBHIRLSOFSA, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.462337151 (563h59m55.815122279s) [noise=true , address=45.91.116.8:4048 , piece_id=KDUUMIWPL56B4KUYAHZ3SVL4E3EHYGB2NCSJKZESBPYNBCABUR7A, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.462702948 (563h59m55.815488076s) [noise=true , address=145.79.227.18:4007 , piece_id=X2ILLKTIZSULE6EL2DKZ2V7WZFWCXZ5Z7NMEQQU7ORG6QANYGE5A, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.463054467 (563h59m55.815839596s) [noise=true , address=45.91.116.4:4939 , piece_id=ZVT77KYOPEZU5NVKQZNJRLJK6AFBI2IGK77KSGRG4WS26GKPC5AA, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.463414943 (563h59m55.816200081s) [noise=true , address=45.91.116.7:4009 , piece_id=HY4OYCNGEFGATDPFK62QZ5OMYM7VIBIGR5DOFXS42D63K2PWHWRQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.464552759 (563h59m55.817337888s) [noise=true , address=45.91.116.6:4975 , piece_id=P2OPDQC5CAUNU6PKCZJ74BHRV4X2ZY7YGZAFSHTWMTLK34ORHHKQ, seg_pos={0 0}, upload=496352]: Uploading piece... Sep 22 02:08:20.570763590 (563h59m55.923548719s) [upload=496352]: Next returned nil segment Sep 22 02:08:20.570779229 (563h59m55.923564358s) [upload=496352]: Waiting for error group managing segments... Sep 22 02:08:20.579340004 (563h59m55.932125133s) [noise=true , address=45.91.116.5:4299 , piece_id=7RLKVP75JC53US6SNGULMLNL2VQSBS4C2JXJ2XEJM5FWQN3JKW6A, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.579408563 (563h59m55.932193691s) [noise=true , address=45.91.116.4:4937 , piece_id=TSG3DWXGOAMIKEFOI3GL266ONTV5O5PZVJIJEJVA3GHZVEFLCWBQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.580552390 (563h59m55.933337517s) [noise=true , address=77.90.31.17:4770 , piece_id=LBBHDCEYUGA5IL3SETBBBXN44RR3MPS6CET4WOVHEHBHIRLSOFSA, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.580668457 (563h59m55.933453586s) [noise=true , address=45.91.116.7:4009 , piece_id=HY4OYCNGEFGATDPFK62QZ5OMYM7VIBIGR5DOFXS42D63K2PWHWRQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.581149310 (563h59m55.933934438s) [noise=true , address=45.91.116.6:4975 , piece_id=P2OPDQC5CAUNU6PKCZJ74BHRV4X2ZY7YGZAFSHTWMTLK34ORHHKQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.581266139 (563h59m55.934051267s) [noise=true , address=45.91.116.8:4048 , piece_id=KDUUMIWPL56B4KUYAHZ3SVL4E3EHYGB2NCSJKZESBPYNBCABUR7A, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.581741721 (563h59m55.934526849s) [noise=true , address=77.90.31.14:4663 , piece_id=6IB3E74QRO2AMLSZ3SOL6FHHCIAD2YCLSB327HX7JSSNMJHZ4YNQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.581957146 (563h59m55.934742274s) [noise=true , address=45.91.116.4:4939 , piece_id=ZVT77KYOPEZU5NVKQZNJRLJK6AFBI2IGK77KSGRG4WS26GKPC5AA, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.582328282 (563h59m55.93511341s) [noise=true , address=77.90.31.10:4518 , piece_id=GSZBFIOCXFVRZNCDBC2DFJUJLYHWYW7MUAS7C4XINUX7IH5KUQDQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.582506817 (563h59m55.935291945s) [noise=true , address=45.91.116.9:4262 , piece_id=NM4FUTBS3RJWOHEVN4KMBL4TEEVBIOOSROBAWZWJKLQTWWJMGIBA, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:20.583549694 (563h59m55.936334832s) [noise=true , address=77.90.31.19:4842 , piece_id=YESLECXUJ5I2NOZXRW6HSET7YNWF6373QNIXCPHWMZHC5TFTYLJA, seg_pos={0 0}, upload=496352]: Done uploading piece. err: <nil> Sep 22 02:08:50.400527190 (564h0m25.753312298s) [noise=true , address=145.79.227.15:4876 , piece_id=XPFBENRLP7GSVALZDUMBAIM5Y4S4MS7LGXLNGMNO3RZMLHSFU3GQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: rpc: dial tcp 145.79.227.15:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:50.400557446 (564h0m25.753342545s) [noise=true , address=145.79.227.7:4295 , piece_id=WMEUSUFV3UET3KJIHNKXSJOIBSWFX7YV2F5Z4PN53WG24TSVYKCQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:08:50.400588505 (564h0m25.753373603s) [noise=true , address=145.79.227.11:4731 , piece_id=IOSKWC6JD4IBOLF2IB443K522B67ET7CAG33VGXXQOMJAUFVZT5A, seg_pos={0 0}, upload=496352]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:08:50.400603342 (564h0m25.75338844s) [noise=true , address=145.79.227.16:4909 , piece_id=P3Z2HGUTNOXDJF4TDR4LGXVWKFKUODFMSMHQGWT4NU3C5UYGXBVQ, seg_pos={0 0}, upload=496352]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:50.400635312 (564h0m25.753420411s) [noise=true , address=145.79.227.18:4007 , piece_id=X2ILLKTIZSULE6EL2DKZ2V7WZFWCXZ5Z7NMEQQU7ORG6QANYGE5A, seg_pos={0 0}, upload=496352]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: rpc: dial tcp 145.79.227.18:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:50.400663836 (564h0m25.753448944s) [seg_pos={0 0}, upload=496352]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:08:50.427527518 (564h0m25.780312636s) [upload=496352]: Done uploading segments... Sep 22 02:08:50.427543308 (564h0m25.780328426s) [upload=496352]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.02830719s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 398.057623ms FAIL (2025-09-22 02:08:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 398.082148ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.00063573s FAIL (2025-09-22 02:08:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000655397s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.030818614s FAIL (2025-09-22 02:08:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:07:50.332589668 (563h59m25.685374797s) [upload=496346]: Starting upload Sep 22 02:07:50.332649120 (563h59m25.685434248s) [upload=496346]: Done starting upload Sep 22 02:07:50.332918215 (563h59m25.685703343s) [upload=496346]: Uploading segments... Sep 22 02:07:50.333007302 (563h59m25.68579243s) [upload=496346]: Got next segment. Inline: false Sep 22 02:07:50.333041406 (563h59m25.685826534s) [upload=496346]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:07:50.333046906 (563h59m25.685832024s) [upload=496346]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:07:50.364608994 (563h59m25.717394122s) [seg_pos={0 0}, upload=496346]: Begin upload segment... Sep 22 02:07:50.364643699 (563h59m25.717428827s) [seg_pos={0 0}, upload=496346]: Done begin upload segment. Sep 22 02:07:50.364670129 (563h59m25.717455256s) [noise=true , address=145.79.227.7:4294 , piece_id=4XRWCYEXNNOFQTXQJFX4ZHPGKMEQSNBCGJ2FWGMWBMXTUB67VXDA, seg_pos={0 0}, upload=496346]: Uploading piece... Sep 22 02:07:50.364696128 (563h59m25.717481255s) [noise=true , address=45.91.116.13:4155 , piece_id=DTYEMRLP5G2OXCXGQWKHXBTOKDY2QHEKBGZNHTNKYRI44MGDO7JA, seg_pos={0 0}, upload=496346]: Uploading piece... Sep 22 02:07:50.365120805 (563h59m25.717905933s) [noise=true , address=45.91.116.10:4083 , piece_id=6AUMHQTHNIVME6MXJM2O43G6S7IHFPBUHTGL45Z4B26MZQYMGTJQ, seg_pos={0 0}, upload=496346]: Uploading piece... Sep 22 02:07:50.365199011 (563h59m25.71798414s) [noise=true , address=77.90.31.12:4592 , piece_id=RTOQNMMRCZLRRZ7YMWIXBDIKKEESJKZKGQ33ZIR2NP3FLGHBROLQ, seg_pos={0 0}, upload=496346]: Uploading piece... Sep 22 02:07:50.365516737 (563h59m25.718301866s) [noise=true , address=145.79.227.15:4874 , piece_id=67UFHWVOGOYMS2CMEHCZOIZT5JBSJL4OSOHFJSE4M2MS2722ZTKA, seg_pos={0 0}, upload=496346]: Uploading piece... Sep 22 02:07:50.365609942 (563h59m25.718395071s) [noise=true , address=45.91.116.5:4297 , piece_id=5ZBLKE3MTNSY2MDRM7QG5AVCMTTU3IUSK6HUQ7ZN6ZEM3CXFFGMA, seg_pos={0 0}, upload=496346]: Uploading piece... Sep 22 02:07:50.365872556 (563h59m25.718657684s) [noise=true , address=145.79.227.11:4730 , piece_id=IK4SFR4DV2PRBYZ4IZK2ZWD3OZZ6J2TJOC3OSDV4I5VRE7XT4MIQ, seg_pos={0 0}, upload=496346]: Uploading piece... Sep 22 02:07:50.366039198 (563h59m25.718824336s) [noise=true , address=45.91.116.11:4117 , piece_id=LRWY45DCEADB6J4YP6MCOM3MYNULCF2NHD2PMM3Z5HCH73ACWSBA, seg_pos={0 0}, upload=496346]: Uploading piece... Sep 22 02:07:50.366271353 (563h59m25.719056482s) [noise=true , address=77.90.31.15:4699 , piece_id=7VXDBJTXEY6NSWN3ZQMLER4O6TFLSFHMANXJZI6K2GIUUEFF6WEA, seg_pos={0 0}, upload=496346]: Uploading piece... Sep 22 02:07:50.366563903 (563h59m25.719349041s) [noise=true , address=45.91.116.7:4011 , piece_id=RPE5454AO5QHG62VVGSP52TCNMWKFLJLNLCDP72SVKE2H7OCFNNQ, seg_pos={0 0}, upload=496346]: Uploading piece... Sep 22 02:07:50.366650476 (563h59m25.719435604s) [noise=true , address=77.90.31.12:4589 , piece_id=226UFZJY5PIL43JREVKVRYAV5QFJJUOEUEWEXEBEELA6NNMFWVLQ, seg_pos={0 0}, upload=496346]: Uploading piece... Sep 22 02:07:50.366962801 (563h59m25.71974793s) [noise=true , address=45.91.116.8:4047 , piece_id=IWJAPF4MDAYIUGU6UZCD5Q2YAZKMLDU47EDDAQFSABNZFZCRJWGQ, seg_pos={0 0}, upload=496346]: Uploading piece... Sep 22 02:07:50.366993119 (563h59m25.719778246s) [noise=true , address=77.90.31.19:4841 , piece_id=SLH66F3JOEENSSY2NQVHOOQFAKON4XWTVKL7K7TK2RP2H27M27EQ, seg_pos={0 0}, upload=496346]: Uploading piece... Sep 22 02:07:50.367314261 (563h59m25.720099399s) [noise=true , address=145.79.227.11:4729 , piece_id=6WIKU4GGNNKKQMXEPY36TLP66PUU73BOKB5XNDR3WYNJOE7WFGXQ, seg_pos={0 0}, upload=496346]: Uploading piece... Sep 22 02:07:50.367340149 (563h59m25.720125278s) [noise=true , address=77.90.31.16:4734 , piece_id=5TCUZNP3QKPRSW27SMFNVXSZF7KXWYHMEVE2Y52SB2MC3437YSPQ, seg_pos={0 0}, upload=496346]: Uploading piece... Sep 22 02:07:50.367715885 (563h59m25.720501013s) [noise=true , address=77.90.31.14:4661 , piece_id=645X56AEP5YWZFX2IHJQYWAGDQAWDOXAZVWRL3TD6KMXY44LPX4Q, seg_pos={0 0}, upload=496346]: Uploading piece... Sep 22 02:07:50.507648959 (563h59m25.860434086s) [upload=496346]: Next returned nil segment Sep 22 02:07:50.507675208 (563h59m25.860460336s) [upload=496346]: Waiting for error group managing segments... Sep 22 02:07:50.516802436 (563h59m25.869587564s) [noise=true , address=45.91.116.5:4297 , piece_id=5ZBLKE3MTNSY2MDRM7QG5AVCMTTU3IUSK6HUQ7ZN6ZEM3CXFFGMA, seg_pos={0 0}, upload=496346]: Done uploading piece. err: <nil> Sep 22 02:07:50.516871385 (563h59m25.869656513s) [noise=true , address=45.91.116.13:4155 , piece_id=DTYEMRLP5G2OXCXGQWKHXBTOKDY2QHEKBGZNHTNKYRI44MGDO7JA, seg_pos={0 0}, upload=496346]: Done uploading piece. err: <nil> Sep 22 02:07:50.518353878 (563h59m25.871139015s) [noise=true , address=45.91.116.7:4011 , piece_id=RPE5454AO5QHG62VVGSP52TCNMWKFLJLNLCDP72SVKE2H7OCFNNQ, seg_pos={0 0}, upload=496346]: Done uploading piece. err: <nil> Sep 22 02:07:50.518568120 (563h59m25.871353247s) [noise=true , address=45.91.116.8:4047 , piece_id=IWJAPF4MDAYIUGU6UZCD5Q2YAZKMLDU47EDDAQFSABNZFZCRJWGQ, seg_pos={0 0}, upload=496346]: Done uploading piece. err: <nil> Sep 22 02:07:50.519368772 (563h59m25.87215391s) [noise=true , address=45.91.116.10:4083 , piece_id=6AUMHQTHNIVME6MXJM2O43G6S7IHFPBUHTGL45Z4B26MZQYMGTJQ, seg_pos={0 0}, upload=496346]: Done uploading piece. err: <nil> Sep 22 02:07:50.519674275 (563h59m25.872459413s) [noise=true , address=45.91.116.11:4117 , piece_id=LRWY45DCEADB6J4YP6MCOM3MYNULCF2NHD2PMM3Z5HCH73ACWSBA, seg_pos={0 0}, upload=496346]: Done uploading piece. err: <nil> Sep 22 02:07:50.520337830 (563h59m25.873122969s) [noise=true , address=77.90.31.12:4589 , piece_id=226UFZJY5PIL43JREVKVRYAV5QFJJUOEUEWEXEBEELA6NNMFWVLQ, seg_pos={0 0}, upload=496346]: Done uploading piece. err: <nil> Sep 22 02:07:50.520583842 (563h59m25.87336897s) [noise=true , address=77.90.31.19:4841 , piece_id=SLH66F3JOEENSSY2NQVHOOQFAKON4XWTVKL7K7TK2RP2H27M27EQ, seg_pos={0 0}, upload=496346]: Done uploading piece. err: <nil> Sep 22 02:07:50.521183808 (563h59m25.873968937s) [noise=true , address=77.90.31.16:4734 , piece_id=5TCUZNP3QKPRSW27SMFNVXSZF7KXWYHMEVE2Y52SB2MC3437YSPQ, seg_pos={0 0}, upload=496346]: Done uploading piece. err: <nil> Sep 22 02:07:50.521390296 (563h59m25.874175433s) [noise=true , address=77.90.31.14:4661 , piece_id=645X56AEP5YWZFX2IHJQYWAGDQAWDOXAZVWRL3TD6KMXY44LPX4Q, seg_pos={0 0}, upload=496346]: Done uploading piece. err: <nil> Sep 22 02:07:50.522037560 (563h59m25.874822698s) [noise=true , address=77.90.31.12:4592 , piece_id=RTOQNMMRCZLRRZ7YMWIXBDIKKEESJKZKGQ33ZIR2NP3FLGHBROLQ, seg_pos={0 0}, upload=496346]: Done uploading piece. err: <nil> Sep 22 02:07:50.522161453 (563h59m25.874946581s) [noise=true , address=77.90.31.15:4699 , piece_id=7VXDBJTXEY6NSWN3ZQMLER4O6TFLSFHMANXJZI6K2GIUUEFF6WEA, seg_pos={0 0}, upload=496346]: Done uploading piece. err: <nil> Sep 22 02:08:20.318658217 (563h59m55.671443345s) [noise=true , address=145.79.227.15:4874 , piece_id=67UFHWVOGOYMS2CMEHCZOIZT5JBSJL4OSOHFJSE4M2MS2722ZTKA, seg_pos={0 0}, upload=496346]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: rpc: dial tcp 145.79.227.15:4874: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:20.318728068 (563h59m55.671513186s) [noise=true , address=145.79.227.11:4730 , piece_id=IK4SFR4DV2PRBYZ4IZK2ZWD3OZZ6J2TJOC3OSDV4I5VRE7XT4MIQ, seg_pos={0 0}, upload=496346]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:20.318788251 (563h59m55.671573378s) [noise=true , address=145.79.227.11:4729 , piece_id=6WIKU4GGNNKKQMXEPY36TLP66PUU73BOKB5XNDR3WYNJOE7WFGXQ, seg_pos={0 0}, upload=496346]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:20.318821713 (563h59m55.671606841s) [noise=true , address=145.79.227.7:4294 , piece_id=4XRWCYEXNNOFQTXQJFX4ZHPGKMEQSNBCGJ2FWGMWBMXTUB67VXDA, seg_pos={0 0}, upload=496346]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: rpc: dial tcp 145.79.227.7:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:20.318871086 (563h59m55.671656214s) [seg_pos={0 0}, upload=496346]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:08:20.347679508 (563h59m55.700464635s) [upload=496346]: Done uploading segments... Sep 22 02:08:20.347708362 (563h59m55.700493489s) [upload=496346]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.030837189s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 811.658491ms FAIL (2025-09-22 02:07:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== atchRequestItem_ObjectBegin Sep 22 02:06:54.193406974 (563h58m29.546192112s) [upload=496330]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:06:54.251112672 (563h58m29.60389781s) [seg_pos={0 0}, upload=496330]: Begin upload segment... Sep 22 02:06:54.251165431 (563h58m29.603950559s) [seg_pos={0 0}, upload=496330]: Done begin upload segment. Sep 22 02:06:54.251202060 (563h58m29.603987188s) [noise=true , address=145.79.227.52:4470 , piece_id=YMWWXFETGTP77OCUAEMUCVY3MO7XVW2ZCGLQYK2AX66AQQ5CQRDA, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.251228800 (563h58m29.604013938s) [noise=true , address=145.79.227.10:4416 , piece_id=NK5XOSVJNEUJVGHVYNXUYYDYO2UK5BLGNGC4GWD4T2D7RKOJ66SA, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.251667514 (563h58m29.604452652s) [noise=true , address=145.79.227.15:4898 , piece_id=CIK4WGXJEXH4I67X4P5MJ2I3S6IY7PE7JH4MARQBMGR3P7VJE7ZQ, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.251696648 (563h58m29.604481777s) [noise=true , address=145.79.227.25:4635 , piece_id=CKVOTR5PZG2LQXND3GS33F2ZGIAVUSFTODKCW5354OFUJBUVY7BQ, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.252085108 (563h58m29.604870235s) [noise=true , address=145.79.227.44:4226 , piece_id=DPCWRV7UPTFLFT43SRC57D4CEXBWNDP7BX6WIIGXJGBOS2ZGAHVA, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.252218608 (563h58m29.605003746s) [noise=true , address=145.79.227.38:4012 , piece_id=ZBEIIQFY45AUPCOSDC262TUFUEC26FFQEGLHUTNSDXL2N6XKTBKQ, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.252467795 (563h58m29.605252923s) [noise=true , address=145.79.227.31:4805 , piece_id=MMIGHOUFVIHQMKC2DC7YNIIJ5ICI7ZOJXNSX4FGQ7DQOFP7I4KYA, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.252669945 (563h58m29.605455074s) [noise=true , address=145.79.227.45:4264 , piece_id=AFBGIFWDGVH5SCOREE27ZOU47AQK7TSOI226664KRR6D6YED75PQ, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.252820167 (563h58m29.605605305s) [noise=true , address=145.79.227.5:4252 , piece_id=3C3UYTQ2PQLGKTEQCWRMAEXM7TQGX47SAMJTIUX4PCQAPGP5KHFQ, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.253057132 (563h58m29.605842259s) [noise=true , address=145.79.227.50:4399 , piece_id=O7SOLF7F2C7IG7X7BSQHPPPVPQI73FZYIV22673L4WRMEO6SODGA, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.253182407 (563h58m29.605967534s) [noise=true , address=145.79.227.2:4117 , piece_id=WNKTMGQTBT2YAQPV3YZTQ5GR7AL3URY5XQTLZPPOVNRW25OCIJYA, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.253538094 (563h58m29.606323223s) [noise=true , address=77.90.31.21:4901 , piece_id=U63UXY24M424BLHAKPGAM7JAV5Y5NUSEA6HQGLRGLJRJZLBAO6LA, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.253590162 (563h58m29.60637529s) [noise=true , address=145.79.227.12:4799 , piece_id=7XYM3KKYDQ4MHQBQVFCF3BARJYWZYFBWJY2U3H6UGRWKSUUSESCA, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.253910603 (563h58m29.606695731s) [noise=true , address=145.79.227.46:4630 , piece_id=WRDELDKF2FBUJ3IKKN2SIPMJBSTXKWZQBXDLL22XBLYOBBD2YGNQ, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.253951740 (563h58m29.606736868s) [noise=true , address=145.79.227.41:4106 , piece_id=JXOSMBTTBMPC3BCFS62M4L2WDAIO7YPFDELWQU3MTBHEWJJGSZ7Q, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.254304432 (563h58m29.60708956s) [noise=true , address=145.79.227.7:4322 , piece_id=5I56PGPCK7N7RG7RUGLZRJY4EOPC6BNAGMO7GIKVTARMTXNTVCGQ, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.254313589 (563h58m29.607098717s) [noise=true , address=145.79.227.32:4851 , piece_id=5YV5PX2S77VYBDAU3SQOCYGOWKJW3QDXFSYNAOLGAXRFKSCTUTKA, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.254799461 (563h58m29.607584589s) [noise=true , address=145.79.227.6:4283 , piece_id=NOAKPPMXUVWUAKYZW552ITZJITH4V7RXQ6XSPVO55IJYIMZIBWVA, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.351207327 (563h58m29.703992435s) [upload=496330]: Next returned nil segment Sep 22 02:06:54.351222495 (563h58m29.704007603s) [upload=496330]: Waiting for error group managing segments... Sep 22 02:06:54.359774974 (563h58m29.712560113s) [noise=true , address=145.79.227.46:4630 , piece_id=WRDELDKF2FBUJ3IKKN2SIPMJBSTXKWZQBXDLL22XBLYOBBD2YGNQ, seg_pos={0 0}, upload=496330]: Done uploading piece. err: <nil> Sep 22 02:06:54.360449160 (563h58m29.713234258s) [noise=true , address=77.90.31.21:4901 , piece_id=U63UXY24M424BLHAKPGAM7JAV5Y5NUSEA6HQGLRGLJRJZLBAO6LA, seg_pos={0 0}, upload=496330]: Done uploading piece. err: <nil> Sep 22 02:06:54.360974125 (563h58m29.713759252s) [noise=true , address=145.79.227.25:4635 , piece_id=CKVOTR5PZG2LQXND3GS33F2ZGIAVUSFTODKCW5354OFUJBUVY7BQ, seg_pos={0 0}, upload=496330]: Done uploading piece. err: <nil> Sep 22 02:06:54.361262717 (563h58m29.714047815s) [noise=true , address=145.79.227.38:4012 , piece_id=ZBEIIQFY45AUPCOSDC262TUFUEC26FFQEGLHUTNSDXL2N6XKTBKQ, seg_pos={0 0}, upload=496330]: Done uploading piece. err: <nil> Sep 22 02:06:54.384664851 (563h58m29.737449959s) [noise=true , address=145.79.227.32:4851 , piece_id=5YV5PX2S77VYBDAU3SQOCYGOWKJW3QDXFSYNAOLGAXRFKSCTUTKA, seg_pos={0 0}, upload=496330]: Done uploading piece. err: <nil> Sep 22 02:06:54.384720296 (563h58m29.737505434s) [noise=true , address=145.79.227.44:4226 , piece_id=DPCWRV7UPTFLFT43SRC57D4CEXBWNDP7BX6WIIGXJGBOS2ZGAHVA, seg_pos={0 0}, upload=496330]: Done uploading piece. err: <nil> Sep 22 02:06:54.385580310 (563h58m29.738365437s) [noise=true , address=145.79.227.50:4399 , piece_id=O7SOLF7F2C7IG7X7BSQHPPPVPQI73FZYIV22673L4WRMEO6SODGA, seg_pos={0 0}, upload=496330]: Done uploading piece. err: <nil> Sep 22 02:06:54.386182350 (563h58m29.738967478s) [noise=true , address=145.79.227.45:4264 , piece_id=AFBGIFWDGVH5SCOREE27ZOU47AQK7TSOI226664KRR6D6YED75PQ, seg_pos={0 0}, upload=496330]: Done uploading piece. err: <nil> Sep 22 02:06:54.389116165 (563h58m29.741901273s) [noise=true , address=145.79.227.31:4805 , piece_id=MMIGHOUFVIHQMKC2DC7YNIIJ5ICI7ZOJXNSX4FGQ7DQOFP7I4KYA, seg_pos={0 0}, upload=496330]: Done uploading piece. err: <nil> Sep 22 02:06:54.389481370 (563h58m29.742266498s) [noise=true , address=145.79.227.41:4106 , piece_id=JXOSMBTTBMPC3BCFS62M4L2WDAIO7YPFDELWQU3MTBHEWJJGSZ7Q, seg_pos={0 0}, upload=496330]: Done uploading piece. err: <nil> Sep 22 02:06:54.403588383 (563h58m29.756373491s) [noise=true , address=145.79.227.15:4898 , piece_id=CIK4WGXJEXH4I67X4P5MJ2I3S6IY7PE7JH4MARQBMGR3P7VJE7ZQ, seg_pos={0 0}, upload=496330]: Done uploading piece. err: <nil> Sep 22 02:06:54.404384437 (563h58m29.757169565s) [noise=true , address=145.79.227.52:4470 , piece_id=YMWWXFETGTP77OCUAEMUCVY3MO7XVW2ZCGLQYK2AX66AQQ5CQRDA, seg_pos={0 0}, upload=496330]: Done uploading piece. err: <nil> Sep 22 02:07:54.166906499 (563h59m29.519691637s) [noise=true , address=145.79.227.12:4799 , piece_id=7XYM3KKYDQ4MHQBQVFCF3BARJYWZYFBWJY2U3H6UGRWKSUUSESCA, seg_pos={0 0}, upload=496330]: Done uploading piece. err: ecclient: upload failed (node:12pMB9xsz6ArzVRzSkh59X8Y7v3w18CzWoVzBRhPdKZTne241v9, address:145.79.227.12:4799): protocol: rpc: dial tcp 145.79.227.12:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:07:54.166973535 (563h59m29.519758663s) [noise=true , address=145.79.227.5:4252 , piece_id=3C3UYTQ2PQLGKTEQCWRMAEXM7TQGX47SAMJTIUX4PCQAPGP5KHFQ, seg_pos={0 0}, upload=496330]: Done uploading piece. err: ecclient: upload failed (node:125gP7pSQx5QXXgE7boHtFAtBqaZ19Ti144TEhWdJYwxCzncvCs, address:145.79.227.5:4252): protocol: rpc: dial tcp 145.79.227.5:4252: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:07:54.167025963 (563h59m29.519811091s) [noise=true , address=145.79.227.10:4416 , piece_id=NK5XOSVJNEUJVGHVYNXUYYDYO2UK5BLGNGC4GWD4T2D7RKOJ66SA, seg_pos={0 0}, upload=496330]: Done uploading piece. err: ecclient: upload failed (node:12Rwbh1TjuvXvNEcTKsS49FMV3YKJUUzea9K8zFdDXns2ZyMcvg, address:145.79.227.10:4416): protocol: rpc: dial tcp 145.79.227.10:4416: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:07:54.167219516 (563h59m29.520004644s) [noise=true , address=145.79.227.2:4117 , piece_id=WNKTMGQTBT2YAQPV3YZTQ5GR7AL3URY5XQTLZPPOVNRW25OCIJYA, seg_pos={0 0}, upload=496330]: Done uploading piece. err: ecclient: upload failed (node:122Htg72f5eUUutf1p3fYQpy1Wkf6AgdaQV83HHYye2n8zm3the, address:145.79.227.2:4117): protocol: rpc: dial tcp 145.79.227.2:4117: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:07:54.167289437 (563h59m29.520074565s) [noise=true , address=145.79.227.6:4283 , piece_id=NOAKPPMXUVWUAKYZW552ITZJITH4V7RXQ6XSPVO55IJYIMZIBWVA, seg_pos={0 0}, upload=496330]: Done uploading piece. err: ecclient: upload failed (node:1ytKjZnmERiZtgJuMJhXG5KFwUQwie27odve4LPE7Pw5hR5ZaU, address:145.79.227.6:4283): protocol: rpc: dial tcp 145.79.227.6:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:07:54.167358186 (563h59m29.520143324s) [noise=true , address=145.79.227.7:4322 , piece_id=5I56PGPCK7N7RG7RUGLZRJY4EOPC6BNAGMO7GIKVTARMTXNTVCGQ, seg_pos={0 0}, upload=496330]: Done uploading piece. err: ecclient: upload failed (node:12WwYqhEYC9QPkGDiSt5bF7izvhpsawWx5dwTqP5zZcPyigs1n3, address:145.79.227.7:4322): protocol: rpc: dial tcp 145.79.227.7:4322: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:07:54.167455068 (563h59m29.520240206s) [seg_pos={0 0}, upload=496330]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 02:07:54.196954186 (563h59m29.549739313s) [upload=496330]: Done uploading segments... Sep 22 02:07:54.197006163 (563h59m29.549791291s) [upload=496330]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 811.671747ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.032566678s FAIL (2025-09-22 02:07:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== atchRequestItem_ObjectBegin Sep 22 02:06:54.193406974 (563h58m29.546192112s) [upload=496330]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:06:54.251112672 (563h58m29.60389781s) [seg_pos={0 0}, upload=496330]: Begin upload segment... Sep 22 02:06:54.251165431 (563h58m29.603950559s) [seg_pos={0 0}, upload=496330]: Done begin upload segment. Sep 22 02:06:54.251202060 (563h58m29.603987188s) [noise=true , address=145.79.227.52:4470 , piece_id=YMWWXFETGTP77OCUAEMUCVY3MO7XVW2ZCGLQYK2AX66AQQ5CQRDA, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.251228800 (563h58m29.604013938s) [noise=true , address=145.79.227.10:4416 , piece_id=NK5XOSVJNEUJVGHVYNXUYYDYO2UK5BLGNGC4GWD4T2D7RKOJ66SA, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.251667514 (563h58m29.604452652s) [noise=true , address=145.79.227.15:4898 , piece_id=CIK4WGXJEXH4I67X4P5MJ2I3S6IY7PE7JH4MARQBMGR3P7VJE7ZQ, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.251696648 (563h58m29.604481777s) [noise=true , address=145.79.227.25:4635 , piece_id=CKVOTR5PZG2LQXND3GS33F2ZGIAVUSFTODKCW5354OFUJBUVY7BQ, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.252085108 (563h58m29.604870235s) [noise=true , address=145.79.227.44:4226 , piece_id=DPCWRV7UPTFLFT43SRC57D4CEXBWNDP7BX6WIIGXJGBOS2ZGAHVA, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.252218608 (563h58m29.605003746s) [noise=true , address=145.79.227.38:4012 , piece_id=ZBEIIQFY45AUPCOSDC262TUFUEC26FFQEGLHUTNSDXL2N6XKTBKQ, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.252467795 (563h58m29.605252923s) [noise=true , address=145.79.227.31:4805 , piece_id=MMIGHOUFVIHQMKC2DC7YNIIJ5ICI7ZOJXNSX4FGQ7DQOFP7I4KYA, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.252669945 (563h58m29.605455074s) [noise=true , address=145.79.227.45:4264 , piece_id=AFBGIFWDGVH5SCOREE27ZOU47AQK7TSOI226664KRR6D6YED75PQ, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.252820167 (563h58m29.605605305s) [noise=true , address=145.79.227.5:4252 , piece_id=3C3UYTQ2PQLGKTEQCWRMAEXM7TQGX47SAMJTIUX4PCQAPGP5KHFQ, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.253057132 (563h58m29.605842259s) [noise=true , address=145.79.227.50:4399 , piece_id=O7SOLF7F2C7IG7X7BSQHPPPVPQI73FZYIV22673L4WRMEO6SODGA, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.253182407 (563h58m29.605967534s) [noise=true , address=145.79.227.2:4117 , piece_id=WNKTMGQTBT2YAQPV3YZTQ5GR7AL3URY5XQTLZPPOVNRW25OCIJYA, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.253538094 (563h58m29.606323223s) [noise=true , address=77.90.31.21:4901 , piece_id=U63UXY24M424BLHAKPGAM7JAV5Y5NUSEA6HQGLRGLJRJZLBAO6LA, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.253590162 (563h58m29.60637529s) [noise=true , address=145.79.227.12:4799 , piece_id=7XYM3KKYDQ4MHQBQVFCF3BARJYWZYFBWJY2U3H6UGRWKSUUSESCA, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.253910603 (563h58m29.606695731s) [noise=true , address=145.79.227.46:4630 , piece_id=WRDELDKF2FBUJ3IKKN2SIPMJBSTXKWZQBXDLL22XBLYOBBD2YGNQ, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.253951740 (563h58m29.606736868s) [noise=true , address=145.79.227.41:4106 , piece_id=JXOSMBTTBMPC3BCFS62M4L2WDAIO7YPFDELWQU3MTBHEWJJGSZ7Q, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.254304432 (563h58m29.60708956s) [noise=true , address=145.79.227.7:4322 , piece_id=5I56PGPCK7N7RG7RUGLZRJY4EOPC6BNAGMO7GIKVTARMTXNTVCGQ, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.254313589 (563h58m29.607098717s) [noise=true , address=145.79.227.32:4851 , piece_id=5YV5PX2S77VYBDAU3SQOCYGOWKJW3QDXFSYNAOLGAXRFKSCTUTKA, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.254799461 (563h58m29.607584589s) [noise=true , address=145.79.227.6:4283 , piece_id=NOAKPPMXUVWUAKYZW552ITZJITH4V7RXQ6XSPVO55IJYIMZIBWVA, seg_pos={0 0}, upload=496330]: Uploading piece... Sep 22 02:06:54.351207327 (563h58m29.703992435s) [upload=496330]: Next returned nil segment Sep 22 02:06:54.351222495 (563h58m29.704007603s) [upload=496330]: Waiting for error group managing segments... Sep 22 02:06:54.359774974 (563h58m29.712560113s) [noise=true , address=145.79.227.46:4630 , piece_id=WRDELDKF2FBUJ3IKKN2SIPMJBSTXKWZQBXDLL22XBLYOBBD2YGNQ, seg_pos={0 0}, upload=496330]: Done uploading piece. err: <nil> Sep 22 02:06:54.360449160 (563h58m29.713234258s) [noise=true , address=77.90.31.21:4901 , piece_id=U63UXY24M424BLHAKPGAM7JAV5Y5NUSEA6HQGLRGLJRJZLBAO6LA, seg_pos={0 0}, upload=496330]: Done uploading piece. err: <nil> Sep 22 02:06:54.360974125 (563h58m29.713759252s) [noise=true , address=145.79.227.25:4635 , piece_id=CKVOTR5PZG2LQXND3GS33F2ZGIAVUSFTODKCW5354OFUJBUVY7BQ, seg_pos={0 0}, upload=496330]: Done uploading piece. err: <nil> Sep 22 02:06:54.361262717 (563h58m29.714047815s) [noise=true , address=145.79.227.38:4012 , piece_id=ZBEIIQFY45AUPCOSDC262TUFUEC26FFQEGLHUTNSDXL2N6XKTBKQ, seg_pos={0 0}, upload=496330]: Done uploading piece. err: <nil> Sep 22 02:06:54.384664851 (563h58m29.737449959s) [noise=true , address=145.79.227.32:4851 , piece_id=5YV5PX2S77VYBDAU3SQOCYGOWKJW3QDXFSYNAOLGAXRFKSCTUTKA, seg_pos={0 0}, upload=496330]: Done uploading piece. err: <nil> Sep 22 02:06:54.384720296 (563h58m29.737505434s) [noise=true , address=145.79.227.44:4226 , piece_id=DPCWRV7UPTFLFT43SRC57D4CEXBWNDP7BX6WIIGXJGBOS2ZGAHVA, seg_pos={0 0}, upload=496330]: Done uploading piece. err: <nil> Sep 22 02:06:54.385580310 (563h58m29.738365437s) [noise=true , address=145.79.227.50:4399 , piece_id=O7SOLF7F2C7IG7X7BSQHPPPVPQI73FZYIV22673L4WRMEO6SODGA, seg_pos={0 0}, upload=496330]: Done uploading piece. err: <nil> Sep 22 02:06:54.386182350 (563h58m29.738967478s) [noise=true , address=145.79.227.45:4264 , piece_id=AFBGIFWDGVH5SCOREE27ZOU47AQK7TSOI226664KRR6D6YED75PQ, seg_pos={0 0}, upload=496330]: Done uploading piece. err: <nil> Sep 22 02:06:54.389116165 (563h58m29.741901273s) [noise=true , address=145.79.227.31:4805 , piece_id=MMIGHOUFVIHQMKC2DC7YNIIJ5ICI7ZOJXNSX4FGQ7DQOFP7I4KYA, seg_pos={0 0}, upload=496330]: Done uploading piece. err: <nil> Sep 22 02:06:54.389481370 (563h58m29.742266498s) [noise=true , address=145.79.227.41:4106 , piece_id=JXOSMBTTBMPC3BCFS62M4L2WDAIO7YPFDELWQU3MTBHEWJJGSZ7Q, seg_pos={0 0}, upload=496330]: Done uploading piece. err: <nil> Sep 22 02:06:54.403588383 (563h58m29.756373491s) [noise=true , address=145.79.227.15:4898 , piece_id=CIK4WGXJEXH4I67X4P5MJ2I3S6IY7PE7JH4MARQBMGR3P7VJE7ZQ, seg_pos={0 0}, upload=496330]: Done uploading piece. err: <nil> Sep 22 02:06:54.404384437 (563h58m29.757169565s) [noise=true , address=145.79.227.52:4470 , piece_id=YMWWXFETGTP77OCUAEMUCVY3MO7XVW2ZCGLQYK2AX66AQQ5CQRDA, seg_pos={0 0}, upload=496330]: Done uploading piece. err: <nil> Sep 22 02:07:54.166906499 (563h59m29.519691637s) [noise=true , address=145.79.227.12:4799 , piece_id=7XYM3KKYDQ4MHQBQVFCF3BARJYWZYFBWJY2U3H6UGRWKSUUSESCA, seg_pos={0 0}, upload=496330]: Done uploading piece. err: ecclient: upload failed (node:12pMB9xsz6ArzVRzSkh59X8Y7v3w18CzWoVzBRhPdKZTne241v9, address:145.79.227.12:4799): protocol: rpc: dial tcp 145.79.227.12:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:07:54.166973535 (563h59m29.519758663s) [noise=true , address=145.79.227.5:4252 , piece_id=3C3UYTQ2PQLGKTEQCWRMAEXM7TQGX47SAMJTIUX4PCQAPGP5KHFQ, seg_pos={0 0}, upload=496330]: Done uploading piece. err: ecclient: upload failed (node:125gP7pSQx5QXXgE7boHtFAtBqaZ19Ti144TEhWdJYwxCzncvCs, address:145.79.227.5:4252): protocol: rpc: dial tcp 145.79.227.5:4252: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:07:54.167025963 (563h59m29.519811091s) [noise=true , address=145.79.227.10:4416 , piece_id=NK5XOSVJNEUJVGHVYNXUYYDYO2UK5BLGNGC4GWD4T2D7RKOJ66SA, seg_pos={0 0}, upload=496330]: Done uploading piece. err: ecclient: upload failed (node:12Rwbh1TjuvXvNEcTKsS49FMV3YKJUUzea9K8zFdDXns2ZyMcvg, address:145.79.227.10:4416): protocol: rpc: dial tcp 145.79.227.10:4416: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:07:54.167219516 (563h59m29.520004644s) [noise=true , address=145.79.227.2:4117 , piece_id=WNKTMGQTBT2YAQPV3YZTQ5GR7AL3URY5XQTLZPPOVNRW25OCIJYA, seg_pos={0 0}, upload=496330]: Done uploading piece. err: ecclient: upload failed (node:122Htg72f5eUUutf1p3fYQpy1Wkf6AgdaQV83HHYye2n8zm3the, address:145.79.227.2:4117): protocol: rpc: dial tcp 145.79.227.2:4117: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:07:54.167289437 (563h59m29.520074565s) [noise=true , address=145.79.227.6:4283 , piece_id=NOAKPPMXUVWUAKYZW552ITZJITH4V7RXQ6XSPVO55IJYIMZIBWVA, seg_pos={0 0}, upload=496330]: Done uploading piece. err: ecclient: upload failed (node:1ytKjZnmERiZtgJuMJhXG5KFwUQwie27odve4LPE7Pw5hR5ZaU, address:145.79.227.6:4283): protocol: rpc: dial tcp 145.79.227.6:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:07:54.167358186 (563h59m29.520143324s) [noise=true , address=145.79.227.7:4322 , piece_id=5I56PGPCK7N7RG7RUGLZRJY4EOPC6BNAGMO7GIKVTARMTXNTVCGQ, seg_pos={0 0}, upload=496330]: Done uploading piece. err: ecclient: upload failed (node:12WwYqhEYC9QPkGDiSt5bF7izvhpsawWx5dwTqP5zZcPyigs1n3, address:145.79.227.7:4322): protocol: rpc: dial tcp 145.79.227.7:4322: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:07:54.167455068 (563h59m29.520240206s) [seg_pos={0 0}, upload=496330]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 02:07:54.196954186 (563h59m29.549739313s) [upload=496330]: Done uploading segments... Sep 22 02:07:54.197006163 (563h59m29.549791291s) [upload=496330]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 1m0.032587297s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.032302901s FAIL (2025-09-22 02:07:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 02:07:20.248168516 (563h58m55.600953654s) [upload=496336]: Starting upload Sep 22 02:07:20.248302577 (563h58m55.601087716s) [upload=496336]: Done starting upload Sep 22 02:07:20.248534573 (563h58m55.6013197s) [upload=496336]: Uploading segments... Sep 22 02:07:20.248819116 (563h58m55.601604245s) [upload=496336]: Got next segment. Inline: false Sep 22 02:07:20.248891101 (563h58m55.601676229s) [upload=496336]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:07:20.248901010 (563h58m55.601686138s) [upload=496336]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:07:20.272875449 (563h58m55.625660567s) [seg_pos={0 0}, upload=496336]: Begin upload segment... Sep 22 02:07:20.272911576 (563h58m55.625696695s) [seg_pos={0 0}, upload=496336]: Done begin upload segment. Sep 22 02:07:20.272938397 (563h58m55.625723515s) [noise=true , address=145.79.227.13:4804 , piece_id=HFUJDZQVZDCZOFZSFJSUL6GQU45UWNERWHOXOYJZRE63MTHZEMGQ, seg_pos={0 0}, upload=496336]: Uploading piece... Sep 22 02:07:20.272996236 (563h58m55.625781363s) [noise=true , address=77.90.31.18:4806 , piece_id=AOOQWZN5WDLDNJYXOSQISTPV3CYUAFBEXWWXGEMTCGIBSBFCIO4Q, seg_pos={0 0}, upload=496336]: Uploading piece... Sep 22 02:07:20.273475045 (563h58m55.626260163s) [noise=true , address=145.79.227.4:4186 , piece_id=MLXYQQWZODAQB6QEX745EFQ366TC7PBVP2O32RIIFT4AHXVKNU4Q, seg_pos={0 0}, upload=496336]: Uploading piece... Sep 22 02:07:20.273603265 (563h58m55.626388393s) [noise=true , address=145.79.227.18:4005 , piece_id=YTMFPZUGX6HZ3FCNI42ML6AXRHYWRWF5Z26KRZCEYIH4OMDL66UA, seg_pos={0 0}, upload=496336]: Uploading piece... Sep 22 02:07:20.273906614 (563h58m55.626691732s) [noise=true , address=145.79.227.14:4837 , piece_id=4JK4XBWYECJN37AFIUCREX7ZFJ3OLAHEXVODHBWODFPRGV5ZEFRQ, seg_pos={0 0}, upload=496336]: Uploading piece... Sep 22 02:07:20.274135734 (563h58m55.626920861s) [noise=true , address=77.90.31.13:4627 , piece_id=BXQ7EQPX64YL225CW6LM6HLWBNNR7PT727PX53MCQWRYYWHAQGBQ, seg_pos={0 0}, upload=496336]: Uploading piece... Sep 22 02:07:20.274326853 (563h58m55.627111971s) [noise=true , address=145.79.227.13:4803 , piece_id=K7MYYTLNBTX5P5UH4DSGPVSWTKEF3MNCEXBBQP3WQLM7VQIYIWLQ, seg_pos={0 0}, upload=496336]: Uploading piece... Sep 22 02:07:20.274612309 (563h58m55.627397426s) [noise=true , address=145.79.227.13:4801 , piece_id=JLBF45VF2SQIXZYARGDQDU2ZTXUY5ZLAZP75CY5F5TPFDTOEG3WA, seg_pos={0 0}, upload=496336]: Uploading piece... Sep 22 02:07:20.274791805 (563h58m55.627576923s) [noise=true , address=77.90.31.10:4519 , piece_id=2VUWVWMIDJZSCSSXJVJ7QTLPQA2NOTBLXINK75OG6FTG464NT3EQ, seg_pos={0 0}, upload=496336]: Uploading piece... Sep 22 02:07:20.275103610 (563h58m55.627888728s) [noise=true , address=45.91.116.14:4190 , piece_id=PWQSZ2MFY52PYRKJRX5IZUZR3DCGKHEDCQGDRYPP6CUPG7FLM5IQ, seg_pos={0 0}, upload=496336]: Uploading piece... Sep 22 02:07:20.275190614 (563h58m55.627975732s) [noise=true , address=45.91.116.8:4047 , piece_id=NZTLRYGVEGEASNJR7K4F4E24D4KCIFV7IXPDOAPX76YPZWPTD7VA, seg_pos={0 0}, upload=496336]: Uploading piece... Sep 22 02:07:20.275607416 (563h58m55.628392534s) [noise=true , address=45.91.116.11:4119 , piece_id=VG5SSLQOMRWGNI2U5PILV6LRI2J3Q5JL6C6LN6XZPGPIDHGSOHGA, seg_pos={0 0}, upload=496336]: Uploading piece... Sep 22 02:07:20.275639606 (563h58m55.628424735s) [noise=true , address=145.79.227.12:4765 , piece_id=5DWS2OVO4V53LJC5NTDC3NTZXBSMWA47SENWT2LJYV6NUYJWX3LA, seg_pos={0 0}, upload=496336]: Uploading piece... Sep 22 02:07:20.276060626 (563h58m55.628845744s) [noise=true , address=45.91.116.9:4262 , piece_id=PFI2N4CEEUTPKJSVVMJMC5PXXP5V32NPPYPN3UWOZWNIZZTQL5BA, seg_pos={0 0}, upload=496336]: Uploading piece... Sep 22 02:07:20.276362633 (563h58m55.629147761s) [noise=true , address=145.79.227.6:4258 , piece_id=7ZJBEIOKBLYCWRD2E5ZBNY27VYFDTZ3PIPV526VXVFRLMVYC3TEA, seg_pos={0 0}, upload=496336]: Uploading piece... Sep 22 02:07:20.276483170 (563h58m55.629268287s) [noise=true , address=77.90.31.17:4772 , piece_id=DGG37BSQYHX3IQTVRVW7JPYVEEMWMCDNBACPKZD4WOHQL5MMD6RA, seg_pos={0 0}, upload=496336]: Uploading piece... Sep 22 02:07:20.349870115 (563h58m55.702655213s) [upload=496336]: Next returned nil segment Sep 22 02:07:20.349884782 (563h58m55.702669881s) [upload=496336]: Waiting for error group managing segments... Sep 22 02:07:20.359828532 (563h58m55.71261363s) [noise=true , address=145.79.227.18:4005 , piece_id=YTMFPZUGX6HZ3FCNI42ML6AXRHYWRWF5Z26KRZCEYIH4OMDL66UA, seg_pos={0 0}, upload=496336]: Done uploading piece. err: <nil> Sep 22 02:07:20.360233593 (563h58m55.713018691s) [noise=true , address=45.91.116.8:4047 , piece_id=NZTLRYGVEGEASNJR7K4F4E24D4KCIFV7IXPDOAPX76YPZWPTD7VA, seg_pos={0 0}, upload=496336]: Done uploading piece. err: <nil> Sep 22 02:07:20.360279669 (563h58m55.713064777s) [noise=true , address=77.90.31.18:4806 , piece_id=AOOQWZN5WDLDNJYXOSQISTPV3CYUAFBEXWWXGEMTCGIBSBFCIO4Q, seg_pos={0 0}, upload=496336]: Done uploading piece. err: <nil> Sep 22 02:07:20.360509109 (563h58m55.713294208s) [noise=true , address=77.90.31.10:4519 , piece_id=2VUWVWMIDJZSCSSXJVJ7QTLPQA2NOTBLXINK75OG6FTG464NT3EQ, seg_pos={0 0}, upload=496336]: Done uploading piece. err: <nil> Sep 22 02:07:20.361380184 (563h58m55.714165292s) [noise=true , address=145.79.227.14:4837 , piece_id=4JK4XBWYECJN37AFIUCREX7ZFJ3OLAHEXVODHBWODFPRGV5ZEFRQ, seg_pos={0 0}, upload=496336]: Done uploading piece. err: <nil> Sep 22 02:07:20.361465044 (563h58m55.714250141s) [noise=true , address=45.91.116.11:4119 , piece_id=VG5SSLQOMRWGNI2U5PILV6LRI2J3Q5JL6C6LN6XZPGPIDHGSOHGA, seg_pos={0 0}, upload=496336]: Done uploading piece. err: <nil> Sep 22 02:07:20.361688873 (563h58m55.714473972s) [noise=true , address=45.91.116.9:4262 , piece_id=PFI2N4CEEUTPKJSVVMJMC5PXXP5V32NPPYPN3UWOZWNIZZTQL5BA, seg_pos={0 0}, upload=496336]: Done uploading piece. err: <nil> Sep 22 02:07:20.362799188 (563h58m55.715584285s) [noise=true , address=77.90.31.13:4627 , piece_id=BXQ7EQPX64YL225CW6LM6HLWBNNR7PT727PX53MCQWRYYWHAQGBQ, seg_pos={0 0}, upload=496336]: Done uploading piece. err: <nil> Sep 22 02:07:20.366416485 (563h58m55.719201583s) [noise=true , address=77.90.31.17:4772 , piece_id=DGG37BSQYHX3IQTVRVW7JPYVEEMWMCDNBACPKZD4WOHQL5MMD6RA, seg_pos={0 0}, upload=496336]: Done uploading piece. err: <nil> Sep 22 02:07:20.367594106 (563h58m55.720379203s) [noise=true , address=45.91.116.14:4190 , piece_id=PWQSZ2MFY52PYRKJRX5IZUZR3DCGKHEDCQGDRYPP6CUPG7FLM5IQ, seg_pos={0 0}, upload=496336]: Done uploading piece. err: <nil> Sep 22 02:07:50.227755631 (563h59m25.580540739s) [noise=true , address=145.79.227.13:4801 , piece_id=JLBF45VF2SQIXZYARGDQDU2ZTXUY5ZLAZP75CY5F5TPFDTOEG3WA, seg_pos={0 0}, upload=496336]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: rpc: dial tcp 145.79.227.13:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:07:50.227847663 (563h59m25.580632771s) [noise=true , address=145.79.227.12:4765 , piece_id=5DWS2OVO4V53LJC5NTDC3NTZXBSMWA47SENWT2LJYV6NUYJWX3LA, seg_pos={0 0}, upload=496336]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: rpc: dial tcp 145.79.227.12:4765: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:07:50.227961477 (563h59m25.580746585s) [noise=true , address=145.79.227.13:4804 , piece_id=HFUJDZQVZDCZOFZSFJSUL6GQU45UWNERWHOXOYJZRE63MTHZEMGQ, seg_pos={0 0}, upload=496336]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:07:50.228036288 (563h59m25.580821396s) [noise=true , address=145.79.227.4:4186 , piece_id=MLXYQQWZODAQB6QEX745EFQ366TC7PBVP2O32RIIFT4AHXVKNU4Q, seg_pos={0 0}, upload=496336]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:07:50.228100498 (563h59m25.580885606s) [noise=true , address=145.79.227.13:4803 , piece_id=K7MYYTLNBTX5P5UH4DSGPVSWTKEF3MNCEXBBQP3WQLM7VQIYIWLQ, seg_pos={0 0}, upload=496336]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: rpc: dial tcp 145.79.227.13:4803: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:07:50.228126787 (563h59m25.580911926s) [noise=true , address=145.79.227.6:4258 , piece_id=7ZJBEIOKBLYCWRD2E5ZBNY27VYFDTZ3PIPV526VXVFRLMVYC3TEA, seg_pos={0 0}, upload=496336]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: rpc: dial tcp 145.79.227.6:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:07:50.228200195 (563h59m25.580985334s) [seg_pos={0 0}, upload=496336]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 02:07:50.258277206 (563h59m25.611062314s) [upload=496336]: Done uploading segments... Sep 22 02:07:50.258294749 (563h59m25.611079857s) [upload=496336]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.032325824s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 3.128071009s FAIL (2025-09-21 17:17:04 UTC) orbiter warning: operation: Commit uplink: metaclient: manager closed: closed: read tcp 5.161.89.252:28198->35.207.121.91:7777: read: connection reset by peer Duration: 3.128092929s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 3.106036096s FAIL (2025-09-21 17:16:45 UTC) orbiter warning: operation: Commit uplink: metaclient: manager closed: closed: read tcp 5.161.89.252:28198->35.207.121.91:7777: read: connection reset by peer Duration: 3.106061485s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 3.02663851s FAIL (2025-09-21 17:16:29 UTC) orbiter warning: operation: Commit uplink: metaclient: manager closed: closed: read tcp 5.161.89.252:28198->35.207.121.91:7777: read: connection reset by peer Duration: 3.026711927s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 3.024350688s FAIL (2025-09-21 17:16:01 UTC) orbiter warning: operation: Commit uplink: metaclient: manager closed: closed: read tcp 5.161.89.252:28198->35.207.121.91:7777: read: connection reset by peer Duration: 3.024376828s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.906466586s FAIL (2025-09-21 17:14:58 UTC) orbiter warning: operation: Commit uplink: metaclient: manager closed: closed: read tcp 5.161.89.252:28198->35.207.121.91:7777: read: connection reset by peer Duration: 2.906491062s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.956857067s FAIL (2025-09-21 17:14:37 UTC) orbiter warning: operation: Commit uplink: metaclient: manager closed: closed: read tcp 5.161.89.252:28198->35.207.121.91:7777: read: connection reset by peer Duration: 2.956951533s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.923455658s FAIL (2025-09-21 17:14:03 UTC) orbiter warning: operation: Commit uplink: metaclient: manager closed: closed: read tcp 5.161.89.252:28198->35.207.121.91:7777: read: connection reset by peer Duration: 2.923478231s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 3.117623462s FAIL (2025-09-21 17:13:10 UTC) orbiter warning: operation: Commit uplink: metaclient: manager closed: closed: read tcp 5.161.89.252:28198->35.207.121.91:7777: read: connection reset by peer Duration: 3.117646656s core uplink eu1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)