Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
699.091886ms
FAIL (2025-10-06 23:46:07 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 699.110451ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.......X.....X...X..........................................] duration:
428.51535ms
FAIL (2025-10-06 23:45:49 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 428.535608ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [..........X......X...X......................................] duration:
439.473113ms
FAIL (2025-10-06 23:45:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 439.486419ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
640.405839ms
FAIL (2025-10-06 23:45:02 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 640.423293ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [................X........X..................................] duration:
429.228419ms
FAIL (2025-10-06 23:44:58 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 429.246742ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [........X.....X...X.........................................] duration:
590.212858ms
FAIL (2025-10-06 23:44:48 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 590.236872ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [...........X......X...X.....................................] duration:
480.053111ms
FAIL (2025-10-06 23:44:42 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 480.068599ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [.................X........X.................................] duration:
551.500307ms
FAIL (2025-10-06 23:44:32 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 551.520585ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.........X.....X...X........................................] duration:
424.521432ms
FAIL (2025-10-06 23:44:26 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 424.543814ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
625.315171ms
FAIL (2025-10-06 23:43:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 625.338596ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [..........X.....X...X.......................................] duration:
830.505711ms
FAIL (2025-10-06 23:43:47 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 830.536077ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [............X......X...X....................................] duration:
401.373552ms
FAIL (2025-10-06 23:43:29 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 401.394392ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [X..........X.....X...X......................................] duration:
435.979677ms
FAIL (2025-10-06 23:43:27 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 436.020623ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [..................X........X................................] duration:
447.26076ms
FAIL (2025-10-06 23:43:24 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 447.290005ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
642.953332ms
FAIL (2025-10-06 23:42:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 642.969351ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [...................X........X...............................] duration:
407.446129ms
FAIL (2025-10-06 23:42:46 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 407.463221ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
694.381366ms
FAIL (2025-10-06 23:42:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 694.620064ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
5 out of 60 is failed (threshold=3) [XX..........X.....X...X.....................................] duration:
765.374921ms
FAIL (2025-10-06 23:42:28 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 765.402703ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.............X......X...X...................................] duration:
665.745657ms
FAIL (2025-10-06 23:42:25 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 665.766706ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
634.859446ms
FAIL (2025-10-06 23:42:06 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 634.87778ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
5 out of 60 is failed (threshold=3) [.XX..........X.....X...X....................................] duration:
515.874968ms
FAIL (2025-10-06 23:41:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 515.899764ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [....................X........X..............................] duration:
394.145499ms
FAIL (2025-10-06 23:41:46 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 394.168642ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
834.248706ms
FAIL (2025-10-06 23:41:37 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 834.268835ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [..............X......X...X..................................] duration:
431.242688ms
FAIL (2025-10-06 23:41:29 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 431.330072ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
5 out of 60 is failed (threshold=3) [..XX..........X.....X...X...................................] duration:
457.326904ms
FAIL (2025-10-06 23:41:20 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 457.359484ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
710.350485ms
FAIL (2025-10-06 23:41:10 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 710.373419ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....................X........X.............................] duration:
427.206594ms
FAIL (2025-10-06 23:41:08 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 427.23165ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
5 out of 60 is failed (threshold=3) [...XX..........X.....X...X..................................] duration:
555.096073ms
FAIL (2025-10-06 23:41:05 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 555.11607ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
5 out of 60 is failed (threshold=3) [....XX..........X.....X...X.................................] duration:
573.958361ms
FAIL (2025-10-06 23:40:49 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 573.981375ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [...............X......X...X.................................] duration:
447.701826ms
FAIL (2025-10-06 23:40:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 447.723298ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [................X......X...X................................] duration:
451.683709ms
FAIL (2025-10-06 23:40:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 451.703526ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
957.987884ms
FAIL (2025-10-06 23:40:03 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 958.00745ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [......................X........X............................] duration:
418.52834ms
FAIL (2025-10-06 23:39:56 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 418.555862ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
5 out of 60 is failed (threshold=3) [.....XX..........X.....X...X................................] duration:
458.414141ms
FAIL (2025-10-06 23:39:36 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 458.432154ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.................X......X...X...............................] duration:
420.906611ms
FAIL (2025-10-06 23:39:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 420.927792ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
5 out of 60 is failed (threshold=3) [......XX..........X.....X...X...............................] duration:
519.564941ms
FAIL (2025-10-06 23:39:17 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 519.592133ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
951.324802ms
FAIL (2025-10-06 23:39:03 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 951.345049ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [..................X......X...X..............................] duration:
448.069056ms
FAIL (2025-10-06 23:38:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 448.091729ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......................X........X...........................] duration:
397.759359ms
FAIL (2025-10-06 23:38:46 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 397.778365ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
623.080538ms
FAIL (2025-10-06 23:38:42 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 623.095115ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [...................X......X...X.............................] duration:
434.374876ms
FAIL (2025-10-06 23:38:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 434.391427ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [........................X........X..........................] duration:
441.25506ms
FAIL (2025-10-06 23:38:10 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 441.287921ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
5 out of 60 is failed (threshold=3) [.......XX..........X.....X...X..............................] duration:
655.225313ms
FAIL (2025-10-06 23:38:03 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 655.246753ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
593.776232ms
FAIL (2025-10-06 23:37:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 593.7968ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........................X........X.........................] duration:
453.457501ms
FAIL (2025-10-06 23:37:36 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 453.484691ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
581.669254ms
FAIL (2025-10-06 23:37:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 581.687218ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [....................X......X...X............................] duration:
494.796329ms
FAIL (2025-10-06 23:37:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 494.808822ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
5 out of 60 is failed (threshold=3) [........XX..........X.....X...X.............................] duration:
465.363151ms
FAIL (2025-10-06 23:36:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 6)
======LOGS======
8s) [seg_pos={0 0}, upload=810623]: Begin upload segment...
Oct 6 23:14:35.231159887 (921h6m10.583944995s) [seg_pos={0 0}, upload=810623]: Done begin upload segment.
Oct 6 23:14:35.231177129 (921h6m10.583962237s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231202467 (921h6m10.583987575s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231445945 (921h6m10.584231042s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231464639 (921h6m10.584249747s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231662370 (921h6m10.584447468s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231680665 (921h6m10.584465773s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231888995 (921h6m10.584674093s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.231977101 (921h6m10.584762198s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232134006 (921h6m10.584919114s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232215128 (921h6m10.585000236s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232343719 (921h6m10.585128817s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232451020 (921h6m10.585236118s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232590893 (921h6m10.585376001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232671273 (921h6m10.585456381s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.232794845 (921h6m10.585579953s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.233021411 (921h6m10.585806519s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Uploading piece...
Oct 6 23:14:35.301739905 (921h6m10.654525013s) [upload=810623]: Next returned nil segment
Oct 6 23:14:35.301751626 (921h6m10.654536734s) [upload=810623]: Waiting for error group managing segments...
Oct 6 23:14:35.308525018 (921h6m10.661310126s) [noise=true , address=145.79.227.5:4221 , piece_id=GX5MVSFGABSJDQLNPQUNBDCKRUGHPPTRJ2Y4KTSJM3GBHDVMWSUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309028252 (921h6m10.66181336s) [noise=true , address=145.79.227.13:4802 , piece_id=U6QBCPLFOT7FKZER4GTZHZONSLGI32ZFKQ6YMHLMXBVT6Q3G3Q5A, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309677310 (921h6m10.662462408s) [noise=true , address=145.79.227.14:4839 , piece_id=YGAV5O7IBCW654XHKAKGKQRQ4PIHOQYNPRKNNSI4LBZ3N2I24HDQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.309896092 (921h6m10.6626812s) [noise=true , address=145.79.227.3:4150 , piece_id=5JCRCAVAZMPJWDWBPIMMTXYESJQYHCBYZV2D5CGCAWDMD73HC26Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310034271 (921h6m10.662819379s) [noise=true , address=145.79.227.17:4970 , piece_id=EQDZAIB72F2JRHKJD444PCM4AG225NFL4BFT6L74E7QSFVHDH7QQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:14:35.310195242 (921h6m10.66298034s) [noise=true , address=145.79.227.2:4113 , piece_id=3B4UQO3A2V5SDEQTEPLDVFD2GFGIIDZI3GTOC4Q7B3MRMSUTFVUQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: <nil>
Oct 6 23:15:05.199595893 (921h6m40.552381001s) [noise=true , address=77.90.31.10:4517 , piece_id=73CMBGB7TGVXWGQC6IUNB54POXYMSHWYRRRV7GTXXZDF76LIAQVQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199636880 (921h6m40.552421988s) [noise=true , address=45.91.116.12:4225 , piece_id=WXDK5FN4PP4LPW3UF6ZLBXWLIZAXLP64BWNGY2DFIKGIFNESOFLQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199832757 (921h6m40.552617885s) [noise=true , address=77.90.31.21:4877 , piece_id=EUY5Q64A7NBYOTT4I5MTH3SZBEDXYJTI42W6VYOTNL4B5NMTDKNQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 5.161.89.252:53308->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199933757 (921h6m40.552718865s) [noise=true , address=77.90.31.18:4805 , piece_id=2B2PP4HY4ILPNQYUCJH3AZCLM2KQJHYV4QYN43OM5BJQFHM2SI4Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:52568->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.199996945 (921h6m40.552782053s) [noise=true , address=77.90.31.21:4879 , piece_id=YMGHKTOSRW6HQSEGA5Z74727YX5PI3LMGNJEU7JQDYL5UGJ2AHMQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:8812->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200046488 (921h6m40.552831617s) [noise=true , address=77.90.31.12:4592 , piece_id=IXYLGHN3BS4KSQVM7MJDL5JBR2NGXB3HBQPS62DGJKFEMJ5EUCYQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200054824 (921h6m40.552839922s) [noise=true , address=45.91.116.9:4262 , piece_id=LZITJ3U4QUE4VQM2NUAZMPJ5SBEBSVEQW427LRKG4NTRU7KKQUQQ, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:12616->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200091874 (921h6m40.552877001s) [noise=true , address=77.90.31.16:4735 , piece_id=VARS5MVTAXS2Y7C2IDCX7XW4Y72G574Y5SK3JN4LFDBDAJYEBU3Q, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:39316->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200106040 (921h6m40.552891148s) [noise=true , address=77.90.31.11:4554 , piece_id=BBSDAQ52W6Z26QH3XXEPJVW34ZSZCSFLREX44Q4TDSGOPXKA6ANA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200139673 (921h6m40.552924771s) [noise=true , address=77.90.31.13:4626 , piece_id=ANQSEX2GFRGK5AHO5IFQYKXB5ABZDICBTKNY4OKOP5DUZCQYS4QA, seg_pos={0 0}, upload=810623]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:18720->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:15:05.200164760 (921h6m40.552949858s) [seg_pos={0 0}, upload=810623]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil>
Oct 6 23:15:05.216712375 (921h6m40.569497483s) [upload=810623]: Done uploading segments...
Oct 6 23:15:05.216730419 (921h6m40.569515517s) [upload=810623]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6)
Duration: 465.430998ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........................X........X........................] duration:
360.754212ms
FAIL (2025-10-06 23:36:36 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 9)
======LOGS======
27420042 (921h10m47.58020515s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227624987 (921h10m47.580410085s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227650294 (921h10m47.580435392s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227876188 (921h10m47.580661287s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.227943646 (921h10m47.580728743s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228144602 (921h10m47.58092971s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228179508 (921h10m47.580964616s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228374473 (921h10m47.581159581s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228432042 (921h10m47.581217139s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228581452 (921h10m47.58136656s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228676600 (921h10m47.581461698s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228792097 (921h10m47.581577206s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228974810 (921h10m47.581759918s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.228999326 (921h10m47.581784434s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229204211 (921h10m47.581989319s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.229215272 (921h10m47.58200038s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Uploading piece...
Oct 6 23:19:12.299081179 (921h10m47.651866277s) [upload=810691]: Next returned nil segment
Oct 6 23:19:12.299097009 (921h10m47.651882117s) [upload=810691]: Waiting for error group managing segments...
Oct 6 23:19:12.306034258 (921h10m47.658819356s) [noise=true , address=145.79.227.49:4370 , piece_id=OJGFH57JPEP3TQMZPHKZZGBXM4T5NVWLY2FASXI3HDH2NAQ42IPQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.307622568 (921h10m47.660407677s) [noise=true , address=145.79.227.23:4567 , piece_id=GYCMVSDGAVCOM4HHNZ6MV5XFXF74ROVIDBEH4T2SI6HHPGSBEKZA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.308054209 (921h10m47.660839317s) [noise=true , address=145.79.227.44:4208 , piece_id=6FULKTE3VS7CUU7XHN63GYR6DLPB2XWLCLPYYHWPONKQXQAR5PPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.316367240 (921h10m47.669152338s) [noise=true , address=145.79.227.27:4716 , piece_id=F623KX4I2W5S7L5LGBI2TKIJM5Y24FFH24BP6PUPYNRZL4TDR77A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.353623418 (921h10m47.706408517s) [noise=true , address=145.79.227.32:4840 , piece_id=5GZ3DEIWB7GZ7Y2DECGJ3PGMI6QC2QBGNJLSK7L3G2VRLQAA2GQA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.798268805 (921h10m48.151053913s) [noise=true , address=64.49.13.170:4727 , piece_id=RS5G3CGE7LP5DOW3SBBBUX3IM4HUDDLHUGXSREXVMH2ZO56UIN5A, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:19:12.819297331 (921h10m48.172082429s) [noise=true , address=64.49.13.138:4490 , piece_id=GLUHKHS4R4TOFLA42GJ6NH7L5HPLD5VIMLGGEIZOAA7D7FE3CVJQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:06.839967999 (921h11m42.192753127s) [noise=true , address=77.90.31.19:4848 , piece_id=ZX4BOHGZ6HVOBRXEYS5F6EC3N7SOB5UGH2CLI54J26TFULTTB7OQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:08.034348426 (921h11m43.387133564s) [noise=true , address=77.90.31.12:4624 , piece_id=ICL7ASYLXD4QDCT5FZ3UE5WIPBNZGGQSUAQ74MK4MERHCYIQ47MQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: <nil>
Oct 6 23:20:12.173936997 (921h11m47.526722105s) [noise=true , address=45.91.116.10:4103 , piece_id=7MWAQ2IA7FAK4NG3VZICDI4CDDLIMWWO5BRQHCCSVM3LCWQLZ5GQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:126w8AaZrrF8Y4uMP5yrGGadnMCCEiuP1s2CKg4XwkA7kPMvvS5, address:45.91.116.10:4103): protocol: rpc: dial tcp 45.91.116.10:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174003862 (921h11m47.52678897s) [noise=true , address=77.90.31.18:4831 , piece_id=KHUNWJJ72OHJVVHUVGMF7FWISBJUFBJQR2WXT2TNP7EWUUPZ4XHQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1NenCLQMH3bdNj59r7KjQdMmwqe96gtBJdbj3faWuM2Ym3BDbA, address:77.90.31.18:4831): protocol: rpc: dial tcp 77.90.31.18:4831: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174067181 (921h11m47.526852289s) [noise=true , address=77.90.31.11:4588 , piece_id=GWSMRI5BA67PKIIMMUB2NPBRNTVSFZEBS3SZ646NRCLCIAJ2YSPA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: rpc: dial tcp 77.90.31.11:4588: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174147411 (921h11m47.526932519s) [noise=true , address=45.91.116.6:4001 , piece_id=L4LE4J4C3IXNJI4NGDPLD3M2LV72U4NO4LYRG3BZJH5TVOOFYYWA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1vQJmv9n2yWXZtCCaABrxkB3NN6vDwLdiBkRibDN3t6YNmcmGP, address:45.91.116.6:4001): protocol: rpc: dial tcp 45.91.116.6:4001: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174295850 (921h11m47.527080958s) [noise=true , address=45.91.116.14:4206 , piece_id=NAQRZQDRRPOPYPAICXJYFWESN2UYEGTMH2H4TR7JSTM3VIOHNHRQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: rpc: dial tcp 45.91.116.14:4206: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174441473 (921h11m47.527226581s) [noise=true , address=77.90.31.14:4669 , piece_id=YZUXVZYTK76CPUTVGTN6V63SIHTIHIAKPCSXLCROPELODS4TXNOA, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: read tcp 5.161.89.252:59908->77.90.31.14:4669: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174480827 (921h11m47.527265935s) [noise=true , address=45.91.116.7:4016 , piece_id=UHX6YREOZ7JEY54I35NIO5JQEJGJPDUKPQV6HA76LPOFME4XDVNQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 5.161.89.252:21556->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174537793 (921h11m47.527322901s) [noise=true , address=77.90.31.11:4559 , piece_id=R34MR6CY445YC23EWBKY2IOGJG2EUIZ62UTEMMANUPUHEZOD5M2Q, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1KER2FucoaWotduZNiFAnbFz7gotJYFofKnyQNCRvWQ8rjVDwY, address:77.90.31.11:4559): protocol: read tcp 5.161.89.252:27818->77.90.31.11:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174593429 (921h11m47.527378537s) [noise=true , address=77.90.31.16:4755 , piece_id=XVJRVIFXDLQF2PKNLAOS2YWWNC52CCJCBLRYN7WAMR3DHZAEDCMQ, seg_pos={0 0}, upload=810691]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: read tcp 5.161.89.252:25238->77.90.31.16:4755: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 6 23:20:12.174612024 (921h11m47.527397132s) [seg_pos={0 0}, upload=810691]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil>
Oct 6 23:20:12.191155500 (921h11m47.543940619s) [upload=810691]: Done uploading segments...
Oct 6 23:20:12.191171450 (921h11m47.543956569s) [upload=810691]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9)
Duration: 360.776404ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
571.030782ms
FAIL (2025-10-06 23:36:29 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 571.046051ms
core
uplink
us-select-1
us1-placement-20
|