Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
621.913987ms
FAIL (2025-10-06 23:55:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 621.935537ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [....X........X..............................................] duration:
379.910843ms
FAIL (2025-10-06 23:55:40 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 379.937232ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
601.673064ms
FAIL (2025-10-06 23:55:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 601.700285ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...X...X....................................................] duration:
389.7681ms
FAIL (2025-10-06 23:55:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 389.787548ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.X...X......................................................] duration:
552.55719ms
FAIL (2025-10-06 23:55:13 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 552.587967ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....X........X.............................................] duration:
393.648333ms
FAIL (2025-10-06 23:55:04 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 393.669372ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [......X........X............................................] duration:
406.807037ms
FAIL (2025-10-06 23:54:48 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 406.826012ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
636.455337ms
FAIL (2025-10-06 23:54:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 636.469273ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [....X...X...................................................] duration:
456.408489ms
FAIL (2025-10-06 23:54:17 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 456.431393ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..X...X.....................................................] duration:
499.529506ms
FAIL (2025-10-06 23:54:07 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 499.548531ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......X........X...........................................] duration:
566.552365ms
FAIL (2025-10-06 23:53:44 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 566.579586ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
772.231864ms
FAIL (2025-10-06 23:53:23 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 772.252472ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....X...X..................................................] duration:
430.623772ms
FAIL (2025-10-06 23:53:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 430.671272ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...X...X....................................................] duration:
617.38537ms
FAIL (2025-10-06 23:52:58 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 617.408594ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [....X...X...................................................] duration:
429.490717ms
FAIL (2025-10-06 23:52:29 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 429.516515ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [........X........X..........................................] duration:
480.996775ms
FAIL (2025-10-06 23:52:29 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 481.018616ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
876.886576ms
FAIL (2025-10-06 23:52:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 876.913728ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....X...X..................................................] duration:
376.874624ms
FAIL (2025-10-06 23:52:09 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 376.898739ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [......X...X.................................................] duration:
453.407ms
FAIL (2025-10-06 23:52:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 453.532526ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
655.468209ms
FAIL (2025-10-06 23:51:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 655.492976ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [X.....X...X.................................................] duration:
466.412575ms
FAIL (2025-10-06 23:51:43 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 466.439336ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........X........X.........................................] duration:
498.45588ms
FAIL (2025-10-06 23:51:43 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 498.494592ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [X......X...X................................................] duration:
388.229966ms
FAIL (2025-10-06 23:51:20 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 388.241259ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
623.688269ms
FAIL (2025-10-06 23:51:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 623.71035ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.X......X...X...............................................] duration:
426.918341ms
FAIL (2025-10-06 23:51:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 426.940474ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........X........X........................................] duration:
399.877055ms
FAIL (2025-10-06 23:50:46 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 399.897804ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.X.....X...X................................................] duration:
469.072112ms
FAIL (2025-10-06 23:50:29 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 469.09776ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [..X......X...X..............................................] duration:
451.652731ms
FAIL (2025-10-06 23:50:14 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 451.676075ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
639.607968ms
FAIL (2025-10-06 23:50:09 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 639.631331ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [...........X........X.......................................] duration:
487.801812ms
FAIL (2025-10-06 23:49:50 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 487.8218ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
586.661279ms
FAIL (2025-10-06 23:49:47 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 586.680646ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [...X......X...X.............................................] duration:
409.483772ms
FAIL (2025-10-06 23:49:35 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 409.693566ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [..X.....X...X...............................................] duration:
398.884312ms
FAIL (2025-10-06 23:49:24 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 398.913196ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [....X......X...X............................................] duration:
379.435501ms
FAIL (2025-10-06 23:49:06 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 379.453155ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
869.82367ms
FAIL (2025-10-06 23:48:47 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 869.842514ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [............X........X......................................] duration:
428.506194ms
FAIL (2025-10-06 23:48:41 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 428.530618ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.....X......X...X...........................................] duration:
449.168591ms
FAIL (2025-10-06 23:48:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 449.255313ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [...X.....X...X..............................................] duration:
561.435742ms
FAIL (2025-10-06 23:48:17 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 561.458444ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [......X......X...X..........................................] duration:
471.225895ms
FAIL (2025-10-06 23:48:06 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 471.243628ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [....X.....X...X.............................................] duration:
734.913403ms
FAIL (2025-10-06 23:47:47 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 734.950442ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
849.546081ms
FAIL (2025-10-06 23:47:42 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 849.569306ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.......X......X...X.........................................] duration:
468.508763ms
FAIL (2025-10-06 23:47:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 468.525104ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [.............X........X.....................................] duration:
452.574832ms
FAIL (2025-10-06 23:47:33 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 452.597435ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
851.447759ms
FAIL (2025-10-06 23:47:14 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 851.622087ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [........X......X...X........................................] duration:
431.313691ms
FAIL (2025-10-06 23:47:10 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 431.460437ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [..............X........X....................................] duration:
679.110357ms
FAIL (2025-10-06 23:47:07 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 679.128401ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.....X.....X...X............................................] duration:
465.62959ms
FAIL (2025-10-06 23:46:58 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 465.646813ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.........X......X...X.......................................] duration:
419.166485ms
FAIL (2025-10-06 23:46:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 419.190741ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [......X.....X...X...........................................] duration:
423.075741ms
FAIL (2025-10-06 23:46:39 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 423.098543ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [...............X........X...................................] duration:
566.421786ms
FAIL (2025-10-06 23:46:08 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 566.441824ms
core
uplink
us-select-1
us1-placement-20
|