Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
6 out of 60 is failed (threshold=3) [...........X..........XX..........X.....X...X...............] duration:
362.829994ms
FAIL (2025-10-06 23:27:08 UTC)
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 362.849971ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..............X.................................X...........] duration:
607.738513ms
FAIL (2025-10-06 23:26:57 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 607.761777ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.................................X......X...X...............] duration:
369.181864ms
FAIL (2025-10-06 23:26:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 369.192996ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [......................................X........X............] duration:
396.296741ms
FAIL (2025-10-06 23:26:43 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 396.334533ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
6 out of 60 is failed (threshold=3) [............X..........XX..........X.....X...X..............] duration:
376.612549ms
FAIL (2025-10-06 23:26:34 UTC)
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 376.637566ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...............X.................................X..........] duration:
587.813662ms
FAIL (2025-10-06 23:26:08 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 587.827136ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [..................................X......X...X..............] duration:
440.194653ms
FAIL (2025-10-06 23:26:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 440.250407ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
6 out of 60 is failed (threshold=3) [.............X..........XX..........X.....X...X.............] duration:
644.013403ms
FAIL (2025-10-06 23:26:02 UTC)
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 644.03279ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......................................X........X...........] duration:
416.163505ms
FAIL (2025-10-06 23:25:28 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 416.203599ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [................X.................................X.........] duration:
641.655535ms
FAIL (2025-10-06 23:25:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 641.674853ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [........................................X........X..........] duration:
552.42608ms
FAIL (2025-10-06 23:25:08 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 552.448501ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [...................................X......X...X.............] duration:
417.419802ms
FAIL (2025-10-06 23:25:03 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 417.468233ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.................X.................................X........] duration:
819.990891ms
FAIL (2025-10-06 23:25:02 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 820.019765ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
6 out of 60 is failed (threshold=3) [..............X..........XX..........X.....X...X............] duration:
428.920568ms
FAIL (2025-10-06 23:24:58 UTC)
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 428.94363ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........................................X........X.........] duration:
476.996371ms
FAIL (2025-10-06 23:24:41 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 477.020205ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..................X.................................X.......] duration:
660.461382ms
FAIL (2025-10-06 23:24:40 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 660.482612ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
6 out of 60 is failed (threshold=3) [...............X..........XX..........X.....X...X...........] duration:
395.135274ms
FAIL (2025-10-06 23:24:19 UTC)
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 395.168295ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........................................X........X........] duration:
593.086127ms
FAIL (2025-10-06 23:24:18 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 593.108038ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [...........................................X........X.......] duration:
555.380211ms
FAIL (2025-10-06 23:23:48 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 555.412582ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [....................................X......X...X............] duration:
417.048748ms
FAIL (2025-10-06 23:23:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 417.061421ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.....................................X......X...X...........] duration:
544.231651ms
FAIL (2025-10-06 23:23:32 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 544.253362ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...................X.................................X......] duration:
607.543047ms
FAIL (2025-10-06 23:23:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 607.553898ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [............................................X........X......] duration:
649.281198ms
FAIL (2025-10-06 23:23:28 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 649.320341ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [......................................X......X...X..........] duration:
428.597448ms
FAIL (2025-10-06 23:23:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 428.659315ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
6 out of 60 is failed (threshold=3) [................X..........XX..........X.....X...X..........] duration:
396.463984ms
FAIL (2025-10-06 23:23:06 UTC)
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 396.481056ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [.............................................X........X.....] duration:
566.625807ms
FAIL (2025-10-06 23:22:57 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 566.652167ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
6 out of 60 is failed (threshold=3) [.................X..........XX..........X.....X...X.........] duration:
655.69298ms
FAIL (2025-10-06 23:22:48 UTC)
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 655.721374ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [....................X.................................X.....] duration:
789.615984ms
FAIL (2025-10-06 23:22:47 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 789.635703ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.......................................X......X...X.........] duration:
472.19032ms
FAIL (2025-10-06 23:22:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 472.209495ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [........................................X......X...X........] duration:
399.325936ms
FAIL (2025-10-06 23:22:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 399.342167ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [..............................................X........X....] duration:
450.974295ms
FAIL (2025-10-06 23:22:10 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 450.992188ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.....................X.................................X....] duration:
635.368869ms
FAIL (2025-10-06 23:22:06 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 635.389788ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
6 out of 60 is failed (threshold=3) [..................X..........XX..........X.....X...X........] duration:
436.373714ms
FAIL (2025-10-06 23:21:48 UTC)
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 436.399402ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [......................X.................................X...] duration:
797.244309ms
FAIL (2025-10-06 23:21:42 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 797.261341ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [...............................................X........X...] duration:
380.66735ms
FAIL (2025-10-06 23:21:23 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 380.689431ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.........................................X......X...X.......] duration:
484.333428ms
FAIL (2025-10-06 23:20:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 484.355921ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [................................................X........X..] duration:
1.183805443s
FAIL (2025-10-06 23:20:50 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 1.183822395s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.......................X.................................X..] duration:
1.172463924s
FAIL (2025-10-06 23:20:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.172663489s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
6 out of 60 is failed (threshold=3) [...................X..........XX..........X.....X...X.......] duration:
439.731965ms
FAIL (2025-10-06 23:20:37 UTC)
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 439.754929ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [........................X.................................X.] duration:
6.787865567s
FAIL (2025-10-06 23:20:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.78787767s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [.................................................X........X.] duration:
513.326421ms
FAIL (2025-10-06 23:20:12 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 513.344614ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.........................X.................................X] duration:
1m0.001254198s
FAIL (2025-10-06 23:20:12 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.001274255s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
6 out of 60 is failed (threshold=3) [....................X..........XX..........X.....X...X......] duration:
15.927906489s
FAIL (2025-10-06 23:20:05 UTC)
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 15.927930584s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [..........................................X......X...X......] duration:
7.692159077s
FAIL (2025-10-06 23:20:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.69218184s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
6 out of 60 is failed (threshold=3) [.....................X..........XX..........X.....X...X.....] duration:
455.557956ms
FAIL (2025-10-06 23:18:50 UTC)
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 455.583925ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [...........................................X......X...X.....] duration:
437.525113ms
FAIL (2025-10-06 23:18:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 437.742411ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
603.230865ms
FAIL (2025-10-06 23:18:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 603.253568ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
399.439039ms
FAIL (2025-10-06 23:18:00 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
:35.809563147 (921h3m11.162348244s) [seg_pos={0 0}, upload=810577]: Begin upload segment...
Oct 6 23:11:35.809587702 (921h3m11.1623728s) [seg_pos={0 0}, upload=810577]: Done begin upload segment.
Oct 6 23:11:35.809607890 (921h3m11.162392988s) [noise=true , address=64.49.13.170:4728 , piece_id=YVQWH6JAW5H2PPDRBPOUEJEP4JEXJMJOFI6ZRA7QBTOY4FRVUUWA, seg_pos={0 0}, upload=810577]: Uploading piece...
Oct 6 23:11:35.809621256 (921h3m11.162406353s) [noise=true , address=77.90.31.15:4717 , piece_id=VKI6VCH2NSWK5ILM7UMVAA4GYXM6VAVHYFLH4ZRYUKS6MLVMOXXA, seg_pos={0 0}, upload=810577]: Uploading piece...
Oct 6 23:11:35.809862639 (921h3m11.162647737s) [noise=true , address=45.91.116.8:4058 , piece_id=UCOVQWF2LMYVXOSJWSTBYUWBWPQWKGDB5BGANPJAP5FCFIQMTUDA, seg_pos={0 0}, upload=810577]: Uploading piece...
Oct 6 23:11:35.809935506 (921h3m11.162720614s) [noise=true , address=145.79.227.46:4636 , piece_id=M3X5SGNDJ2W2DFXMHYRCZC2U6KP65WFOPAXOE4OV2ZJXQ3JZ6U5Q, seg_pos={0 0}, upload=810577]: Uploading piece...
Oct 6 23:11:35.810072883 (921h3m11.162857982s) [noise=true , address=77.90.31.17:4793 , piece_id=2RHB45CNKCHCJBO2JFXDUW7LTRHB4ZXVO4AU2HK2ZJMHTDD7LWOQ, seg_pos={0 0}, upload=810577]: Uploading piece...
Oct 6 23:11:35.810201284 (921h3m11.162986382s) [noise=true , address=77.90.31.13:4629 , piece_id=MEFBFMFDLYEDAUCX5Z57CZPPXF23PJ7SPOOS6VBWRDARZZR7WYNQ, seg_pos={0 0}, upload=810577]: Uploading piece...
Oct 6 23:11:35.810333482 (921h3m11.16311859s) [noise=true , address=145.79.227.22:4542 , piece_id=GQJC3V5EVADLPSIIZX3WLFMDF4MGSOLIBOUPNLKMK6M5S6TU32ZQ, seg_pos={0 0}, upload=810577]: Uploading piece...
Oct 6 23:11:35.810503220 (921h3m11.163288328s) [noise=true , address=45.91.116.10:4086 , piece_id=N7NDOOXZBKQESXBSOYOS6PMJ2ZPNVKX2UPTCGFQRMIJXGGGYARSQ, seg_pos={0 0}, upload=810577]: Uploading piece...
Oct 6 23:11:35.810550219 (921h3m11.163335317s) [noise=true , address=64.49.13.138:4483 , piece_id=RO2BRPFRTTOEKRSK46U57UOOU23OID2MHATKKER22REAW6LZ77OQ, seg_pos={0 0}, upload=810577]: Uploading piece...
Oct 6 23:11:35.810752629 (921h3m11.163537737s) [noise=true , address=45.91.116.4:4969 , piece_id=OPFYPTUJ6GVURZ4VI6TOWJ4ZTVNZRVRNYXU7EILEXRDRTNEPZA2A, seg_pos={0 0}, upload=810577]: Uploading piece...
Oct 6 23:11:35.810765003 (921h3m11.163550111s) [noise=true , address=145.79.227.25:4638 , piece_id=HWMQUW7VPAPTYF7VNIZTKQMXTFAS5QHU5LEVVMGGSBT33L4QBHSQ, seg_pos={0 0}, upload=810577]: Uploading piece...
Oct 6 23:11:35.810957553 (921h3m11.163742652s) [noise=true , address=64.49.13.26:4954 , piece_id=5ALOFIE25F4GXUMOOMCDUGHLC3VJWOGGZY4GGHS4KPYZVPVL5PRQ, seg_pos={0 0}, upload=810577]: Uploading piece...
Oct 6 23:11:35.811023277 (921h3m11.163808385s) [noise=true , address=77.90.31.10:4543 , piece_id=YSSNJK5GCFP6DRT52L3IFSAEKCXUYEI4JVEPTMZCKP5HZ2SSHNRQ, seg_pos={0 0}, upload=810577]: Uploading piece...
Oct 6 23:11:35.811167938 (921h3m11.163953036s) [noise=true , address=145.79.227.39:4034 , piece_id=O23VFHC7FMIZS3QAR2IGDTMOPH76CRDIVUBNAKYW6WYTGVFUU6YA, seg_pos={0 0}, upload=810577]: Uploading piece...
Oct 6 23:11:35.811293383 (921h3m11.164078491s) [noise=true , address=145.79.227.43:4180 , piece_id=46G7IFIZYMPA7O23LLXUPVMF4Z2Z355GRVOBQSXTMBEEVF47PHKA, seg_pos={0 0}, upload=810577]: Uploading piece...
Oct 6 23:11:35.811417035 (921h3m11.164202134s) [noise=true , address=145.79.227.52:4467 , piece_id=72BPXV7MQ6HL6IYVRJY5ZYNZXBZK5DANY7DYUDHZN3JOXMMR335A, seg_pos={0 0}, upload=810577]: Uploading piece...
Oct 6 23:11:35.811637500 (921h3m11.164422608s) [noise=true , address=145.79.227.27:4710 , piece_id=MGHHT2XK7KWLVD7VN7Z66MMOJNRWHDNDS7BC7BWEPGJWANSUTMUA, seg_pos={0 0}, upload=810577]: Uploading piece...
Oct 6 23:11:35.811840040 (921h3m11.164625138s) [noise=true , address=145.79.227.35:4487 , piece_id=IZ6PDJZ3PBYENUXSLIRF77B26GY3HR7PJNV2G5YSYBCMU4QFUM6A, seg_pos={0 0}, upload=810577]: Uploading piece...
Oct 6 23:11:35.876127968 (921h3m11.228913076s) [upload=810577]: Next returned nil segment
Oct 6 23:11:35.876140171 (921h3m11.228925269s) [upload=810577]: Waiting for error group managing segments...
Oct 6 23:11:35.883432457 (921h3m11.236217565s) [noise=true , address=145.79.227.39:4034 , piece_id=O23VFHC7FMIZS3QAR2IGDTMOPH76CRDIVUBNAKYW6WYTGVFUU6YA, seg_pos={0 0}, upload=810577]: Done uploading piece. err: <nil>
Oct 6 23:11:35.883754090 (921h3m11.236539198s) [noise=true , address=145.79.227.43:4180 , piece_id=46G7IFIZYMPA7O23LLXUPVMF4Z2Z355GRVOBQSXTMBEEVF47PHKA, seg_pos={0 0}, upload=810577]: Done uploading piece. err: <nil>
Oct 6 23:11:35.883964114 (921h3m11.236749212s) [noise=true , address=145.79.227.25:4638 , piece_id=HWMQUW7VPAPTYF7VNIZTKQMXTFAS5QHU5LEVVMGGSBT33L4QBHSQ, seg_pos={0 0}, upload=810577]: Done uploading piece. err: <nil>
Oct 6 23:11:35.885045895 (921h3m11.237831003s) [noise=true , address=145.79.227.22:4542 , piece_id=GQJC3V5EVADLPSIIZX3WLFMDF4MGSOLIBOUPNLKMK6M5S6TU32ZQ, seg_pos={0 0}, upload=810577]: Done uploading piece. err: <nil>
Oct 6 23:11:35.885329446 (921h3m11.238114545s) [noise=true , address=145.79.227.27:4710 , piece_id=MGHHT2XK7KWLVD7VN7Z66MMOJNRWHDNDS7BC7BWEPGJWANSUTMUA, seg_pos={0 0}, upload=810577]: Done uploading piece. err: <nil>
Oct 6 23:11:35.885434994 (921h3m11.238220103s) [noise=true , address=145.79.227.52:4467 , piece_id=72BPXV7MQ6HL6IYVRJY5ZYNZXBZK5DANY7DYUDHZN3JOXMMR335A, seg_pos={0 0}, upload=810577]: Done uploading piece. err: <nil>
Oct 6 23:11:35.885700733 (921h3m11.238485831s) [noise=true , address=145.79.227.35:4487 , piece_id=IZ6PDJZ3PBYENUXSLIRF77B26GY3HR7PJNV2G5YSYBCMU4QFUM6A, seg_pos={0 0}, upload=810577]: Done uploading piece. err: <nil>
Oct 6 23:11:35.886007770 (921h3m11.238792878s) [noise=true , address=145.79.227.46:4636 , piece_id=M3X5SGNDJ2W2DFXMHYRCZC2U6KP65WFOPAXOE4OV2ZJXQ3JZ6U5Q, seg_pos={0 0}, upload=810577]: Done uploading piece. err: <nil>
Oct 6 23:11:36.386975923 (921h3m11.739761052s) [noise=true , address=64.49.13.170:4728 , piece_id=YVQWH6JAW5H2PPDRBPOUEJEP4JEXJMJOFI6ZRA7QBTOY4FRVUUWA, seg_pos={0 0}, upload=810577]: Done uploading piece. err: <nil>
Oct 6 23:11:36.400346556 (921h3m11.753131684s) [noise=true , address=64.49.13.138:4483 , piece_id=RO2BRPFRTTOEKRSK46U57UOOU23OID2MHATKKER22REAW6LZ77OQ, seg_pos={0 0}, upload=810577]: Done uploading piece. err: <nil>
Oct 6 23:11:36.430989544 (921h3m11.783774671s) [noise=true , address=64.49.13.26:4954 , piece_id=5ALOFIE25F4GXUMOOMCDUGHLC3VJWOGGZY4GGHS4KPYZVPVL5PRQ, seg_pos={0 0}, upload=810577]: Done uploading piece. err: <nil>
Oct 6 23:12:32.124753979 (921h4m7.477539087s) [noise=true , address=45.91.116.4:4969 , piece_id=OPFYPTUJ6GVURZ4VI6TOWJ4ZTVNZRVRNYXU7EILEXRDRTNEPZA2A, seg_pos={0 0}, upload=810577]: Done uploading piece. err: <nil>
Oct 6 23:12:35.776881675 (921h4m11.129666802s) [noise=true , address=77.90.31.15:4717 , piece_id=VKI6VCH2NSWK5ILM7UMVAA4GYXM6VAVHYFLH4ZRYUKS6MLVMOXXA, seg_pos={0 0}, upload=810577]: Done uploading piece. err: ecclient: upload failed (node:12ULANtxKNaJviAZFsmHjDQWjG4x2wfmjCwbQtHmn9LpnELmkPc, address:77.90.31.15:4717): protocol: write tcp 5.161.89.252:21828->77.90.31.15:4717: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:12:35.776907392 (921h4m11.129692521s) [noise=true , address=45.91.116.10:4086 , piece_id=N7NDOOXZBKQESXBSOYOS6PMJ2ZPNVKX2UPTCGFQRMIJXGGGYARSQ, seg_pos={0 0}, upload=810577]: Done uploading piece. err: ecclient: upload failed (node:1X6eg8EWzh7BPYUaLN1wGYPDfkg1RCKZYqDCcJhCtQXApbsG5A, address:45.91.116.10:4086): protocol: read tcp 5.161.89.252:46224->45.91.116.10:4086: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:12:35.776967846 (921h4m11.129752964s) [noise=true , address=77.90.31.10:4543 , piece_id=YSSNJK5GCFP6DRT52L3IFSAEKCXUYEI4JVEPTMZCKP5HZ2SSHNRQ, seg_pos={0 0}, upload=810577]: Done uploading piece. err: ecclient: upload failed (node:12hFnVoQD97rytjEVNtPo8QumC37Eq1RM9MeXBP6Bap9rhK73bX, address:77.90.31.10:4543): protocol: read tcp 5.161.89.252:36754->77.90.31.10:4543: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:12:35.777013391 (921h4m11.12979852s) [noise=true , address=77.90.31.13:4629 , piece_id=MEFBFMFDLYEDAUCX5Z57CZPPXF23PJ7SPOOS6VBWRDARZZR7WYNQ, seg_pos={0 0}, upload=810577]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: read tcp 5.161.89.252:50000->77.90.31.13:4629: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:12:35.777031185 (921h4m11.129816313s) [noise=true , address=77.90.31.17:4793 , piece_id=2RHB45CNKCHCJBO2JFXDUW7LTRHB4ZXVO4AU2HK2ZJMHTDD7LWOQ, seg_pos={0 0}, upload=810577]: Done uploading piece. err: ecclient: upload failed (node:1rxrH2F4LTfrK9SMsNNEuqVr4AjV1VbEYDc3veWbQf88rj2Es4, address:77.90.31.17:4793): protocol: write tcp 5.161.89.252:34804->77.90.31.17:4793: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:12:35.777073133 (921h4m11.129858262s) [noise=true , address=45.91.116.8:4058 , piece_id=UCOVQWF2LMYVXOSJWSTBYUWBWPQWKGDB5BGANPJAP5FCFIQMTUDA, seg_pos={0 0}, upload=810577]: Done uploading piece. err: ecclient: upload failed (node:12GHuEEAGNMcY8EM4n2kFnvtA7p46ZzNHSZYw7yaXdTRCU1Kq6B, address:45.91.116.8:4058): protocol: read tcp 5.161.89.252:36286->45.91.116.8:4058: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:12:35.777092450 (921h4m11.129877578s) [seg_pos={0 0}, upload=810577]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Oct 6 23:12:35.790072881 (921h4m11.142858008s) [upload=810577]: Done uploading segments...
Oct 6 23:12:35.790116883 (921h4m11.142902011s) [upload=810577]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 399.462071ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
6 out of 60 is failed (threshold=3) [......................X..........XX..........X.....X...X....] duration:
442.151988ms
FAIL (2025-10-06 23:17:55 UTC)
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 442.177896ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [............................................X......X...X....] duration:
461.711108ms
FAIL (2025-10-06 23:17:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 461.73359ms
edge
gateway
us-select-1
|