Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 116995 reports (Page 1686 of 2340)
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................]
duration:
209.110213ms
FAIL (2025-08-19 04:58:07 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:26171->127.0.0.1:53: i/o timeout
Duration: 209.123639ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
6 out of 60 is failed (threshold=3) [..............XXXXXX........................................]
duration:
520.613045ms
FAIL (2025-08-19 04:58:05 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
ad=261108]: Uploading piece...
Aug 19 04:27:59.298715587 (251h1m40.029983236s) [noise=true , address=45.91.116.10:4081    , piece_id=VXW7T7RYDQYFA67EV4IP5EFMNLSPHO6YMYO4NCUBCDOSPQWID6KQ, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.298767684 (251h1m40.030035364s) [noise=true , address=45.91.116.11:4119    , piece_id=6NZDOUCLI5JRQ5XXZMNLBXIVXMHR7NJY6BTCMA22QMNC3ZJJFCMA, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.299257736 (251h1m40.030525387s) [noise=true , address=45.91.116.10:4083    , piece_id=N7ABFK4IGEFBUNEAQI3SFS7IC6SCOHQD4TN2QOWDUSNRM4ZLXV7A, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.299339650 (251h1m40.030607301s) [noise=true , address=77.90.31.21:4878     , piece_id=X3FKRFUU6S3AB6ZPUNPB2HOGLJ5E7U4AWTEFH6PISRIBECP3GVSQ, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.299714204 (251h1m40.030981855s) [noise=true , address=45.91.116.8:4045     , piece_id=OH3KNS4URAIN3T4FLAPSYC2SHQKOEBLJMF2AVG7MXJULLC3P7QYA, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.299820754 (251h1m40.031088415s) [noise=true , address=77.90.31.10:4517     , piece_id=QXH4JBJTSA4VLHPS2OUAR3XF7QEL3WSX2JJT7VTZMRBVWS3OIOQQ, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.300136579 (251h1m40.031404229s) [noise=true , address=77.90.31.12:4592     , piece_id=25GKL2MHGGV6IC7JDU5MBJ4MOCCNPGMXQPYXH7A4FFFJ5EZGVZ2Q, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.300491886 (251h1m40.031759547s) [noise=true , address=45.91.116.9:4261     , piece_id=RAEAWIKMHIV4VGOZY57ZK36BNZGZ2ATXTXC4YHRABT6VWB6MGILA, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.300854618 (251h1m40.032122269s) [noise=true , address=104.36.82.2:4732     , piece_id=WKRORDL2XPRF2KRKZ55EQ2PTPJHPJSQ6NQ4D3VVFK6COGG57KDNQ, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.301277543 (251h1m40.032545194s) [noise=true , address=45.91.116.14:4190    , piece_id=SGZNGKGYEKPU4GXCULPKYIOONZZXZT6UTO7MDWYOJRJJW4WSG3VA, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.301655164 (251h1m40.032922814s) [noise=true , address=104.36.82.7:4910     , piece_id=R6GR56GOKT2K3AX75OL3BWODTZ4FJY27MIOQWFYJVY4UGR66DIBQ, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.302072699 (251h1m40.033340349s) [noise=true , address=104.36.82.16:4294    , piece_id=GT3KQOCWD2YTWBQDAO5QJKYIMZINUF4MESKP6F3P6J73R4IY3NDQ, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:28:02.354368898 (251h1m43.085636557s) [noise=true , address=77.90.31.21:4878     , piece_id=X3FKRFUU6S3AB6ZPUNPB2HOGLJ5E7U4AWTEFH6PISRIBECP3GVSQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: write tcp 5.161.89.252:64042->77.90.31.21:4878: write: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:02.385567243 (251h1m43.116834893s) [noise=true , address=77.90.31.12:4592     , piece_id=25GKL2MHGGV6IC7JDU5MBJ4MOCCNPGMXQPYXH7A4FFFJ5EZGVZ2Q, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 5.161.89.252:34306->77.90.31.12:4592: write: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op
Aug 19 04:28:29.225676153 (251h2m9.956943773s) [upload=261108]: Next segment err: context deadline exceeded
Aug 19 04:28:29.225998790 (251h2m9.95726642s) [noise=true , address=104.36.82.2:4732     , piece_id=WKRORDL2XPRF2KRKZ55EQ2PTPJHPJSQ6NQ4D3VVFK6COGG57KDNQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226039707 (251h2m9.957307326s) [noise=true , address=45.91.116.11:4119    , piece_id=6NZDOUCLI5JRQ5XXZMNLBXIVXMHR7NJY6BTCMA22QMNC3ZJJFCMA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226072177 (251h2m9.957339798s) [noise=true , address=45.91.116.8:4045     , piece_id=OH3KNS4URAIN3T4FLAPSYC2SHQKOEBLJMF2AVG7MXJULLC3P7QYA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226223943 (251h2m9.957491573s) [noise=true , address=45.91.116.14:4190    , piece_id=SGZNGKGYEKPU4GXCULPKYIOONZZXZT6UTO7MDWYOJRJJW4WSG3VA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226274027 (251h2m9.957541647s) [noise=true , address=45.91.116.13:4154    , piece_id=WM56E23Z7FLRF33HYDXONXHODJDHPCJFWGVC3NI7RUTZUESWO4MQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226310786 (251h2m9.957578416s) [noise=true , address=77.90.31.17:4771     , piece_id=3BVZK3ZL7N5FC3OJGFFYTVEG7INVMLMG2QBPXX3UC3EJVXHBGW3Q, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226348096 (251h2m9.957615726s) [noise=true , address=104.36.82.11:4114    , piece_id=P4RJRUZPRTFPRK5WWHKUVEKJKXGPPPCIV4RPFIZOBO2CPOXSTSIQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: rpc: dial tcp 104.36.82.11:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226392119 (251h2m9.957659749s) [noise=true , address=45.91.116.10:4081    , piece_id=VXW7T7RYDQYFA67EV4IP5EFMNLSPHO6YMYO4NCUBCDOSPQWID6KQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 5.161.89.252:44604->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226421564 (251h2m9.957689195s) [noise=true , address=45.91.116.9:4261     , piece_id=RAEAWIKMHIV4VGOZY57ZK36BNZGZ2ATXTXC4YHRABT6VWB6MGILA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226443746 (251h2m9.957711377s) [noise=true , address=45.91.116.4:4940     , piece_id=GYF7CH3463KOWU7DE7WKLYTXYXLTRMPOIK4K3D64ANKOERN33FGQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 5.161.89.252:53922->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226474684 (251h2m9.957742315s) [noise=true , address=104.36.82.7:4910     , piece_id=R6GR56GOKT2K3AX75OL3BWODTZ4FJY27MIOQWFYJVY4UGR66DIBQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226493810 (251h2m9.95776143s) [noise=true , address=45.91.116.10:4083    , piece_id=N7ABFK4IGEFBUNEAQI3SFS7IC6SCOHQD4TN2QOWDUSNRM4ZLXV7A, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: rpc: dial tcp 45.91.116.10:4083: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226509199 (251h2m9.957776829s) [noise=true , address=104.36.82.16:4294    , piece_id=GT3KQOCWD2YTWBQDAO5QJKYIMZINUF4MESKP6F3P6J73R4IY3NDQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:104.36.82.16:4294): protocol: write tcp 5.161.89.252:25886->104.36.82.16:4294: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226527894 (251h2m9.957795524s) [noise=true , address=77.90.31.10:4517     , piece_id=QXH4JBJTSA4VLHPS2OUAR3XF7QEL3WSX2JJT7VTZMRBVWS3OIOQQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226544845 (251h2m9.957812476s) [seg_pos={0 0}, upload=261108]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 520.636519ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
5 out of 60 is failed (threshold=3) [..................XXXX.................X....................]
duration:
559.303077ms
FAIL (2025-08-19 04:57:58 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 19 04:42:23.281867426 (251h16m4.013135076s) [upload=261352]: Starting upload
Aug 19 04:42:23.281936796 (251h16m4.013204446s) [upload=261352]: Done starting upload
Aug 19 04:42:23.282094062 (251h16m4.013361712s) [upload=261352]: Uploading segments...
Aug 19 04:42:23.282376082 (251h16m4.013643742s) [upload=261352]: Got next segment. Inline: false
Aug 19 04:42:23.282443619 (251h16m4.013711269s) [upload=261352]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:42:23.282451122 (251h16m4.013718773s) [upload=261352]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:42:23.284629308 (251h16m4.015896958s) [upload=261352]: Next segment err: metaclient: Too Many Requests
Aug 19 04:42:23.284645568 (251h16m4.015913219s) [upload=261352]: Done uploading segments...
Aug 19 04:42:23.284659365 (251h16m4.015927014s) [upload=261352]: Upload finished. err: metaclient: Too Many Requests
Duration: 559.329838ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)
antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................]
duration:
114.119172ms
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-08-19 04:57:58 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:41477->127.0.0.1:53: i/o timeout
Duration: 114.136324ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................]
duration:
691.078693ms
target:
https://ap1.storj.io/login
FAIL (2025-08-19 04:57:58 UTC)
orbiter warning: Get "https://ap1.storj.io/login": dial tcp 35.213.185.92:443: i/o timeout
Duration: 691.111476ms core satellite ap1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................]
duration:
913.489495ms
FAIL (2025-08-19 04:57:52 UTC)
orbiter warning: operation: StatBucket
uplink: bucket: metaclient: context deadline exceeded
Duration: 913.511907ms core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
4 out of 60 is failed (threshold=3) [.................XXXX.......................................]
duration:
607.478203ms
FAIL (2025-08-19 04:57:51 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 7)
======LOGS======
g piece...
Aug 19 04:28:13.563886368 (251h1m54.295154018s) [noise=true , address=104.36.82.5:4843     , piece_id=Q3SLO2FDTYYYYOQZARVS3SHAPM5J3GCLNFQ355QAV7U5MWKMZQQA, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.564531581 (251h1m54.295799231s) [noise=true , address=45.91.116.7:4037     , piece_id=B42RTCUIHRS5OUFJ5S5E6QWRKEN5H34IMBURUUSSUL53Y2FUYKZQ, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.564716169 (251h1m54.295983828s) [noise=true , address=104.36.82.15:4289    , piece_id=GVBKZHWTPFDGYCM6U5EUYLBOTGCTTLPDWW2WZKZZEVNG4UZ3XEYA, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.565358756 (251h1m54.296626407s) [noise=true , address=77.90.31.21:4891     , piece_id=G57I4V2AOHDBI77GIIULZXTGCFC2BP6CQRX25KA7UNNIJ3WTAGPQ, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.565475757 (251h1m54.296743417s) [noise=true , address=45.91.116.9:4286     , piece_id=KUMIYTCCEU7XS2NMUTPXDSD7TMNASOBWGNWNDD7L6DMEAPXVIFWA, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.566262315 (251h1m54.297529975s) [noise=true , address=104.36.82.44:4880    , piece_id=ZRDRV5AWRCYSUZJANVESK3XUCMLQQXF33WPV6QPX6HAZBYHXOUUQ, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.566506594 (251h1m54.297774285s) [noise=true , address=45.91.116.12:4243    , piece_id=GNLESUJ3FCKKUC7HOR7Z532H7LQT6ZOTOKLBGRN2R6UCJL4S3VFQ, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.567576856 (251h1m54.298844507s) [noise=true , address=45.91.116.4:4952     , piece_id=ZXPM75JCQGD3XQ3BTIP4O6SOLR6XCFNBUYQKKD56RTFVCMEGFWDQ, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.575982240 (251h1m54.30724989s) [noise=true , address=77.90.31.19:4862     , piece_id=7CQOCX2FYKPQCTCBFMN7BMTD3FE2D2X4YIHFLCL22AOSYLKUO33A, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.576556950 (251h1m54.3078246s) [noise=true , address=77.90.31.17:4801     , piece_id=A4D6GQYALBOB46EE7D2KI6422NES4M43E4DTX4CKBLNNP2VHZUJQ, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.577052442 (251h1m54.308320102s) [noise=true , address=104.36.82.13:4204    , piece_id=LWFS4W2N7K6SWYKXYHZRJACAYYDBIB7OGVETZWKVTWWWJBZPS33A, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.577571397 (251h1m54.308839058s) [noise=true , address=104.36.82.12:4156    , piece_id=RMBTQYXVS5NKE2ON3WW5T2CBE6S7S6T74VVVLMUSM5C44UXGRAYA, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.581420654 (251h1m54.312688304s) [noise=true , address=77.90.31.12:4596     , piece_id=AEOFPZ64F5FUTWHLMKFVKQGNGYJYOIU7OZPLYGS7ULR52F5XSNYQ, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.582252959 (251h1m54.313520609s) [noise=true , address=77.90.31.18:4835     , piece_id=W6FEWYN4QCZC5IQFUG2YXSGKHSKKFRGETXK76LFKAVACLA37H7PA, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.672688665 (251h1m54.403956365s) [upload=261114]: Next returned nil segment
Aug 19 04:28:13.672710426 (251h1m54.403978086s) [upload=261114]: Waiting for error group managing segments...
Aug 19 04:28:13.681638350 (251h1m54.412906s) [noise=true , address=79.127.219.48:4085   , piece_id=AT2YETVN3DCMAPBL45XWQQCTW5FR6JC4PWN3CRIMZPQW5OG3RKKQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil>
Aug 19 04:28:28.955286140 (251h2m9.68655377s) [noise=true , address=77.90.31.12:4596     , piece_id=AEOFPZ64F5FUTWHLMKFVKQGNGYJYOIU7OZPLYGS7ULR52F5XSNYQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1xgzDjjKfA3YT4tjGq6Esw9xfhA4xWMWnDbyAmNFUcP85qyFGf, address:77.90.31.12:4596): protocol: rpc: dial tcp 77.90.31.12:4596: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:45.285087265 (251h2m26.016354885s) [noise=true , address=104.36.82.60:4354    , piece_id=76HWEIKV5672IF34IRITARBMKGTN7OJU6GHQZZCDAP2ZUR2JNJFA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil>
Aug 19 04:28:45.286367592 (251h2m26.017635222s) [noise=true , address=104.36.82.13:4204    , piece_id=LWFS4W2N7K6SWYKXYHZRJACAYYDBIB7OGVETZWKVTWWWJBZPS33A, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil>
Aug 19 04:28:45.286429859 (251h2m26.017697489s) [noise=true , address=104.36.82.12:4156    , piece_id=RMBTQYXVS5NKE2ON3WW5T2CBE6S7S6T74VVVLMUSM5C44UXGRAYA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil>
Aug 19 04:28:45.288231957 (251h2m26.019499578s) [noise=true , address=104.36.82.15:4289    , piece_id=GVBKZHWTPFDGYCM6U5EUYLBOTGCTTLPDWW2WZKZZEVNG4UZ3XEYA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil>
Aug 19 04:28:45.316902561 (251h2m26.048170191s) [noise=true , address=104.36.82.44:4880    , piece_id=ZRDRV5AWRCYSUZJANVESK3XUCMLQQXF33WPV6QPX6HAZBYHXOUUQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil>
Aug 19 04:28:46.178003461 (251h2m26.90927112s) [noise=true , address=104.36.82.5:4843     , piece_id=Q3SLO2FDTYYYYOQZARVS3SHAPM5J3GCLNFQ355QAV7U5MWKMZQQA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil>
Aug 19 04:29:13.506175578 (251h2m54.237443199s) [noise=true , address=45.91.116.6:4988     , piece_id=X36K53BYKX53WMZKQWN2FFKCKTHNLIY3KLLPXMQN2UBKZM2CTT6Q, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1TrcoCHhAJeEykP9hJNpygoPSdgFV5v6GzL48JpyNPiAj5ZJRS, address:45.91.116.6:4988): protocol: rpc: dial tcp 45.91.116.6:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506222607 (251h2m54.237490227s) [noise=true , address=45.91.116.9:4286     , piece_id=KUMIYTCCEU7XS2NMUTPXDSD7TMNASOBWGNWNDD7L6DMEAPXVIFWA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:19DfbHig9tzF4VRHtSWkExwUS2bdZ8GFgKnAD9wvMBG7qaesNa, address:45.91.116.9:4286): protocol: rpc: dial tcp 45.91.116.9:4286: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506254087 (251h2m54.237521717s) [noise=true , address=45.91.116.12:4243    , piece_id=GNLESUJ3FCKKUC7HOR7Z532H7LQT6ZOTOKLBGRN2R6UCJL4S3VFQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1e9VEMmhYq6PxqvTpxsoUHzqYVwujfj1UG7HSZFeiu8CderMpM, address:45.91.116.12:4243): protocol: rpc: dial tcp 45.91.116.12:4243: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506625304 (251h2m54.237892934s) [noise=true , address=77.90.31.21:4891     , piece_id=G57I4V2AOHDBI77GIIULZXTGCFC2BP6CQRX25KA7UNNIJ3WTAGPQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:17RDAerfx8AW5NsgWLTXLNDbuAEWQFT3SVM4iwdmuBzWrML83g, address:77.90.31.21:4891): protocol: rpc: dial tcp 77.90.31.21:4891: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506656563 (251h2m54.237924213s) [noise=true , address=77.90.31.19:4862     , piece_id=7CQOCX2FYKPQCTCBFMN7BMTD3FE2D2X4YIHFLCL22AOSYLKUO33A, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1Tvmo3nSgBPcFYW9MmZQGUeMoD3m3ykJHwVMj6tj69iQKgX3Mz, address:77.90.31.19:4862): protocol: rpc: dial tcp 77.90.31.19:4862: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded
Aug 19 04:29:13.506689415 (251h2m54.237957045s) [noise=true , address=77.90.31.18:4835     , piece_id=W6FEWYN4QCZC5IQFUG2YXSGKHSKKFRGETXK76LFKAVACLA37H7PA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:12hAQp2Zr7WSaQSTCZf5pwSyuBHHSrZWq5EVz5hFvSA5ddWebqg, address:77.90.31.18:4835): protocol: write tcp 5.161.89.252:27410->77.90.31.18:4835: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506740311 (251h2m54.23800793s) [noise=true , address=45.91.116.7:4037     , piece_id=B42RTCUIHRS5OUFJ5S5E6QWRKEN5H34IMBURUUSSUL53Y2FUYKZQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1iNPwE2XSfXcaXAXcrH2jcyNGYeDiiWvzAcDQffrmjPwTVqVmK, address:45.91.116.7:4037): protocol: rpc: dial tcp 45.91.116.7:4037: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506774885 (251h2m54.238042535s) [noise=true , address=77.90.31.17:4801     , piece_id=A4D6GQYALBOB46EE7D2KI6422NES4M43E4DTX4CKBLNNP2VHZUJQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:127QRLNwZLcR5gnAgZrCjSUsH5T4jESmZ4kWeAnpj9YxaTSjoYe, address:77.90.31.17:4801): protocol: write tcp 5.161.89.252:48394->77.90.31.17:4801: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506808788 (251h2m54.238076419s) [noise=true , address=45.91.116.4:4952     , piece_id=ZXPM75JCQGD3XQ3BTIP4O6SOLR6XCFNBUYQKKD56RTFVCMEGFWDQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1PcGuaN6uLz1HVzMgBp4vViKJw2KrmaFatL1w9DHgiYqccMUJ9, address:45.91.116.4:4952): protocol: write tcp 5.161.89.252:61300->45.91.116.4:4952: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506825761 (251h2m54.23809341s) [noise=true , address=77.90.31.13:4645     , piece_id=TGFZGBBRZLA32ZJS3BGT7OSHJW4S34PWMYSQKCZT4RG2IUTGPZSQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg, address:77.90.31.13:4645): protocol: rpc: dial tcp 77.90.31.13:4645: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506906624 (251h2m54.238174313s) [seg_pos={0 0}, upload=261114]: Done waiting for segment. successful: 7 optimal: 14 errs: <nil>
Aug 19 04:29:13.523807608 (251h2m54.255075238s) [upload=261114]: Done uploading segments...
Aug 19 04:29:13.523824049 (251h2m54.255091679s) [upload=261114]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 7)
Duration: 607.505294ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................]
duration:
4.80106663s
FAIL (2025-08-19 04:57:47 UTC)
orbiter warning: operation: StatBucket
uplink: bucket: metaclient: context deadline exceeded
Duration: 4.801092419s core uplink ap1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)
antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................]
duration:
808.765982ms
FAIL (2025-08-19 04:57:40 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:34390->127.0.0.1:53: i/o timeout
Duration: 808.789217ms edge authservice http bgp eu1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)
antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................]
duration:
343.983666ms
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318
FAIL (2025-08-19 04:57:34 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:41477->127.0.0.1:53: i/o timeout
Duration: 344.005497ms edge gateway vhost bgp eu1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................]
duration:
348.05701ms
FAIL (2025-08-19 04:57:33 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:26171->127.0.0.1:53: i/o timeout
Duration: 348.087166ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................]
duration:
1.342859556s
FAIL (2025-08-19 04:57:31 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:34390->127.0.0.1:53: i/o timeout
Duration: 1.34287743s core uplink us-select-1 us1-placement-20
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)
antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................]
duration:
89.574398ms
FAIL (2025-08-19 04:57:23 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:26171->127.0.0.1:53: i/o timeout
Duration: 89.599505ms edge authservice http us-select-1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1)
antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................]
duration:
868.742136ms
FAIL (2025-08-19 04:57:22 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:34390->127.0.0.1:53: i/o timeout
Duration: 868.772232ms edge authservice http bgp ap1
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................]
duration:
697.428938ms
target:
https://ap1.storj.io/login
FAIL (2025-08-19 04:57:20 UTC)
orbiter warning: Get "https://ap1.storj.io/login": dial tcp 35.213.185.92:443: i/o timeout
Duration: 697.45061ms core satellite ap1
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing)
antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................]
duration:
8.403505ms
target:
http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt
FAIL (2025-08-19 04:57:13 UTC)
orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp: lookup link.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:34965->127.0.0.1:53: i/o timeout
Duration: 8.419976ms edge linksharing us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
6 out of 60 is failed (threshold=3) [...............XXXXXX.......................................]
duration:
522.801511ms
FAIL (2025-08-19 04:57:10 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
ad=261108]: Uploading piece...
Aug 19 04:27:59.298715587 (251h1m40.029983236s) [noise=true , address=45.91.116.10:4081    , piece_id=VXW7T7RYDQYFA67EV4IP5EFMNLSPHO6YMYO4NCUBCDOSPQWID6KQ, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.298767684 (251h1m40.030035364s) [noise=true , address=45.91.116.11:4119    , piece_id=6NZDOUCLI5JRQ5XXZMNLBXIVXMHR7NJY6BTCMA22QMNC3ZJJFCMA, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.299257736 (251h1m40.030525387s) [noise=true , address=45.91.116.10:4083    , piece_id=N7ABFK4IGEFBUNEAQI3SFS7IC6SCOHQD4TN2QOWDUSNRM4ZLXV7A, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.299339650 (251h1m40.030607301s) [noise=true , address=77.90.31.21:4878     , piece_id=X3FKRFUU6S3AB6ZPUNPB2HOGLJ5E7U4AWTEFH6PISRIBECP3GVSQ, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.299714204 (251h1m40.030981855s) [noise=true , address=45.91.116.8:4045     , piece_id=OH3KNS4URAIN3T4FLAPSYC2SHQKOEBLJMF2AVG7MXJULLC3P7QYA, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.299820754 (251h1m40.031088415s) [noise=true , address=77.90.31.10:4517     , piece_id=QXH4JBJTSA4VLHPS2OUAR3XF7QEL3WSX2JJT7VTZMRBVWS3OIOQQ, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.300136579 (251h1m40.031404229s) [noise=true , address=77.90.31.12:4592     , piece_id=25GKL2MHGGV6IC7JDU5MBJ4MOCCNPGMXQPYXH7A4FFFJ5EZGVZ2Q, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.300491886 (251h1m40.031759547s) [noise=true , address=45.91.116.9:4261     , piece_id=RAEAWIKMHIV4VGOZY57ZK36BNZGZ2ATXTXC4YHRABT6VWB6MGILA, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.300854618 (251h1m40.032122269s) [noise=true , address=104.36.82.2:4732     , piece_id=WKRORDL2XPRF2KRKZ55EQ2PTPJHPJSQ6NQ4D3VVFK6COGG57KDNQ, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.301277543 (251h1m40.032545194s) [noise=true , address=45.91.116.14:4190    , piece_id=SGZNGKGYEKPU4GXCULPKYIOONZZXZT6UTO7MDWYOJRJJW4WSG3VA, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.301655164 (251h1m40.032922814s) [noise=true , address=104.36.82.7:4910     , piece_id=R6GR56GOKT2K3AX75OL3BWODTZ4FJY27MIOQWFYJVY4UGR66DIBQ, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.302072699 (251h1m40.033340349s) [noise=true , address=104.36.82.16:4294    , piece_id=GT3KQOCWD2YTWBQDAO5QJKYIMZINUF4MESKP6F3P6J73R4IY3NDQ, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:28:02.354368898 (251h1m43.085636557s) [noise=true , address=77.90.31.21:4878     , piece_id=X3FKRFUU6S3AB6ZPUNPB2HOGLJ5E7U4AWTEFH6PISRIBECP3GVSQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: write tcp 5.161.89.252:64042->77.90.31.21:4878: write: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:02.385567243 (251h1m43.116834893s) [noise=true , address=77.90.31.12:4592     , piece_id=25GKL2MHGGV6IC7JDU5MBJ4MOCCNPGMXQPYXH7A4FFFJ5EZGVZ2Q, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 5.161.89.252:34306->77.90.31.12:4592: write: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op
Aug 19 04:28:29.225676153 (251h2m9.956943773s) [upload=261108]: Next segment err: context deadline exceeded
Aug 19 04:28:29.225998790 (251h2m9.95726642s) [noise=true , address=104.36.82.2:4732     , piece_id=WKRORDL2XPRF2KRKZ55EQ2PTPJHPJSQ6NQ4D3VVFK6COGG57KDNQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226039707 (251h2m9.957307326s) [noise=true , address=45.91.116.11:4119    , piece_id=6NZDOUCLI5JRQ5XXZMNLBXIVXMHR7NJY6BTCMA22QMNC3ZJJFCMA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226072177 (251h2m9.957339798s) [noise=true , address=45.91.116.8:4045     , piece_id=OH3KNS4URAIN3T4FLAPSYC2SHQKOEBLJMF2AVG7MXJULLC3P7QYA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226223943 (251h2m9.957491573s) [noise=true , address=45.91.116.14:4190    , piece_id=SGZNGKGYEKPU4GXCULPKYIOONZZXZT6UTO7MDWYOJRJJW4WSG3VA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226274027 (251h2m9.957541647s) [noise=true , address=45.91.116.13:4154    , piece_id=WM56E23Z7FLRF33HYDXONXHODJDHPCJFWGVC3NI7RUTZUESWO4MQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226310786 (251h2m9.957578416s) [noise=true , address=77.90.31.17:4771     , piece_id=3BVZK3ZL7N5FC3OJGFFYTVEG7INVMLMG2QBPXX3UC3EJVXHBGW3Q, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226348096 (251h2m9.957615726s) [noise=true , address=104.36.82.11:4114    , piece_id=P4RJRUZPRTFPRK5WWHKUVEKJKXGPPPCIV4RPFIZOBO2CPOXSTSIQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: rpc: dial tcp 104.36.82.11:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226392119 (251h2m9.957659749s) [noise=true , address=45.91.116.10:4081    , piece_id=VXW7T7RYDQYFA67EV4IP5EFMNLSPHO6YMYO4NCUBCDOSPQWID6KQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 5.161.89.252:44604->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226421564 (251h2m9.957689195s) [noise=true , address=45.91.116.9:4261     , piece_id=RAEAWIKMHIV4VGOZY57ZK36BNZGZ2ATXTXC4YHRABT6VWB6MGILA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226443746 (251h2m9.957711377s) [noise=true , address=45.91.116.4:4940     , piece_id=GYF7CH3463KOWU7DE7WKLYTXYXLTRMPOIK4K3D64ANKOERN33FGQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 5.161.89.252:53922->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226474684 (251h2m9.957742315s) [noise=true , address=104.36.82.7:4910     , piece_id=R6GR56GOKT2K3AX75OL3BWODTZ4FJY27MIOQWFYJVY4UGR66DIBQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226493810 (251h2m9.95776143s) [noise=true , address=45.91.116.10:4083    , piece_id=N7ABFK4IGEFBUNEAQI3SFS7IC6SCOHQD4TN2QOWDUSNRM4ZLXV7A, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: rpc: dial tcp 45.91.116.10:4083: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226509199 (251h2m9.957776829s) [noise=true , address=104.36.82.16:4294    , piece_id=GT3KQOCWD2YTWBQDAO5QJKYIMZINUF4MESKP6F3P6J73R4IY3NDQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:104.36.82.16:4294): protocol: write tcp 5.161.89.252:25886->104.36.82.16:4294: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226527894 (251h2m9.957795524s) [noise=true , address=77.90.31.10:4517     , piece_id=QXH4JBJTSA4VLHPS2OUAR3XF7QEL3WSX2JJT7VTZMRBVWS3OIOQQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226544845 (251h2m9.957812476s) [seg_pos={0 0}, upload=261108]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 522.826237ms core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)
antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................]
duration:
114.224449ms
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-08-19 04:57:02 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:41477->127.0.0.1:53: i/o timeout
Duration: 114.258954ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................]
duration:
877.136535ms
FAIL (2025-08-19 04:57:01 UTC)
orbiter warning: operation: StatBucket
uplink: bucket: metaclient: context deadline exceeded
Duration: 877.173836ms core uplink eu1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................]
duration:
173.2165ms
FAIL (2025-08-19 04:56:58 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:26171->127.0.0.1:53: i/o timeout
Duration: 173.289727ms edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)
antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................]
duration:
37.646127ms
FAIL (2025-08-19 04:56:57 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:26171->127.0.0.1:53: i/o timeout
Duration: 37.69036ms edge authservice http us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
4 out of 60 is failed (threshold=3) [..................XXXX......................................]
duration:
567.061086ms
FAIL (2025-08-19 04:56:53 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 7)
======LOGS======
g piece...
Aug 19 04:28:13.563886368 (251h1m54.295154018s) [noise=true , address=104.36.82.5:4843     , piece_id=Q3SLO2FDTYYYYOQZARVS3SHAPM5J3GCLNFQ355QAV7U5MWKMZQQA, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.564531581 (251h1m54.295799231s) [noise=true , address=45.91.116.7:4037     , piece_id=B42RTCUIHRS5OUFJ5S5E6QWRKEN5H34IMBURUUSSUL53Y2FUYKZQ, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.564716169 (251h1m54.295983828s) [noise=true , address=104.36.82.15:4289    , piece_id=GVBKZHWTPFDGYCM6U5EUYLBOTGCTTLPDWW2WZKZZEVNG4UZ3XEYA, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.565358756 (251h1m54.296626407s) [noise=true , address=77.90.31.21:4891     , piece_id=G57I4V2AOHDBI77GIIULZXTGCFC2BP6CQRX25KA7UNNIJ3WTAGPQ, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.565475757 (251h1m54.296743417s) [noise=true , address=45.91.116.9:4286     , piece_id=KUMIYTCCEU7XS2NMUTPXDSD7TMNASOBWGNWNDD7L6DMEAPXVIFWA, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.566262315 (251h1m54.297529975s) [noise=true , address=104.36.82.44:4880    , piece_id=ZRDRV5AWRCYSUZJANVESK3XUCMLQQXF33WPV6QPX6HAZBYHXOUUQ, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.566506594 (251h1m54.297774285s) [noise=true , address=45.91.116.12:4243    , piece_id=GNLESUJ3FCKKUC7HOR7Z532H7LQT6ZOTOKLBGRN2R6UCJL4S3VFQ, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.567576856 (251h1m54.298844507s) [noise=true , address=45.91.116.4:4952     , piece_id=ZXPM75JCQGD3XQ3BTIP4O6SOLR6XCFNBUYQKKD56RTFVCMEGFWDQ, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.575982240 (251h1m54.30724989s) [noise=true , address=77.90.31.19:4862     , piece_id=7CQOCX2FYKPQCTCBFMN7BMTD3FE2D2X4YIHFLCL22AOSYLKUO33A, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.576556950 (251h1m54.3078246s) [noise=true , address=77.90.31.17:4801     , piece_id=A4D6GQYALBOB46EE7D2KI6422NES4M43E4DTX4CKBLNNP2VHZUJQ, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.577052442 (251h1m54.308320102s) [noise=true , address=104.36.82.13:4204    , piece_id=LWFS4W2N7K6SWYKXYHZRJACAYYDBIB7OGVETZWKVTWWWJBZPS33A, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.577571397 (251h1m54.308839058s) [noise=true , address=104.36.82.12:4156    , piece_id=RMBTQYXVS5NKE2ON3WW5T2CBE6S7S6T74VVVLMUSM5C44UXGRAYA, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.581420654 (251h1m54.312688304s) [noise=true , address=77.90.31.12:4596     , piece_id=AEOFPZ64F5FUTWHLMKFVKQGNGYJYOIU7OZPLYGS7ULR52F5XSNYQ, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.582252959 (251h1m54.313520609s) [noise=true , address=77.90.31.18:4835     , piece_id=W6FEWYN4QCZC5IQFUG2YXSGKHSKKFRGETXK76LFKAVACLA37H7PA, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.672688665 (251h1m54.403956365s) [upload=261114]: Next returned nil segment
Aug 19 04:28:13.672710426 (251h1m54.403978086s) [upload=261114]: Waiting for error group managing segments...
Aug 19 04:28:13.681638350 (251h1m54.412906s) [noise=true , address=79.127.219.48:4085   , piece_id=AT2YETVN3DCMAPBL45XWQQCTW5FR6JC4PWN3CRIMZPQW5OG3RKKQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil>
Aug 19 04:28:28.955286140 (251h2m9.68655377s) [noise=true , address=77.90.31.12:4596     , piece_id=AEOFPZ64F5FUTWHLMKFVKQGNGYJYOIU7OZPLYGS7ULR52F5XSNYQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1xgzDjjKfA3YT4tjGq6Esw9xfhA4xWMWnDbyAmNFUcP85qyFGf, address:77.90.31.12:4596): protocol: rpc: dial tcp 77.90.31.12:4596: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:45.285087265 (251h2m26.016354885s) [noise=true , address=104.36.82.60:4354    , piece_id=76HWEIKV5672IF34IRITARBMKGTN7OJU6GHQZZCDAP2ZUR2JNJFA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil>
Aug 19 04:28:45.286367592 (251h2m26.017635222s) [noise=true , address=104.36.82.13:4204    , piece_id=LWFS4W2N7K6SWYKXYHZRJACAYYDBIB7OGVETZWKVTWWWJBZPS33A, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil>
Aug 19 04:28:45.286429859 (251h2m26.017697489s) [noise=true , address=104.36.82.12:4156    , piece_id=RMBTQYXVS5NKE2ON3WW5T2CBE6S7S6T74VVVLMUSM5C44UXGRAYA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil>
Aug 19 04:28:45.288231957 (251h2m26.019499578s) [noise=true , address=104.36.82.15:4289    , piece_id=GVBKZHWTPFDGYCM6U5EUYLBOTGCTTLPDWW2WZKZZEVNG4UZ3XEYA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil>
Aug 19 04:28:45.316902561 (251h2m26.048170191s) [noise=true , address=104.36.82.44:4880    , piece_id=ZRDRV5AWRCYSUZJANVESK3XUCMLQQXF33WPV6QPX6HAZBYHXOUUQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil>
Aug 19 04:28:46.178003461 (251h2m26.90927112s) [noise=true , address=104.36.82.5:4843     , piece_id=Q3SLO2FDTYYYYOQZARVS3SHAPM5J3GCLNFQ355QAV7U5MWKMZQQA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil>
Aug 19 04:29:13.506175578 (251h2m54.237443199s) [noise=true , address=45.91.116.6:4988     , piece_id=X36K53BYKX53WMZKQWN2FFKCKTHNLIY3KLLPXMQN2UBKZM2CTT6Q, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1TrcoCHhAJeEykP9hJNpygoPSdgFV5v6GzL48JpyNPiAj5ZJRS, address:45.91.116.6:4988): protocol: rpc: dial tcp 45.91.116.6:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506222607 (251h2m54.237490227s) [noise=true , address=45.91.116.9:4286     , piece_id=KUMIYTCCEU7XS2NMUTPXDSD7TMNASOBWGNWNDD7L6DMEAPXVIFWA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:19DfbHig9tzF4VRHtSWkExwUS2bdZ8GFgKnAD9wvMBG7qaesNa, address:45.91.116.9:4286): protocol: rpc: dial tcp 45.91.116.9:4286: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506254087 (251h2m54.237521717s) [noise=true , address=45.91.116.12:4243    , piece_id=GNLESUJ3FCKKUC7HOR7Z532H7LQT6ZOTOKLBGRN2R6UCJL4S3VFQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1e9VEMmhYq6PxqvTpxsoUHzqYVwujfj1UG7HSZFeiu8CderMpM, address:45.91.116.12:4243): protocol: rpc: dial tcp 45.91.116.12:4243: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506625304 (251h2m54.237892934s) [noise=true , address=77.90.31.21:4891     , piece_id=G57I4V2AOHDBI77GIIULZXTGCFC2BP6CQRX25KA7UNNIJ3WTAGPQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:17RDAerfx8AW5NsgWLTXLNDbuAEWQFT3SVM4iwdmuBzWrML83g, address:77.90.31.21:4891): protocol: rpc: dial tcp 77.90.31.21:4891: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506656563 (251h2m54.237924213s) [noise=true , address=77.90.31.19:4862     , piece_id=7CQOCX2FYKPQCTCBFMN7BMTD3FE2D2X4YIHFLCL22AOSYLKUO33A, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1Tvmo3nSgBPcFYW9MmZQGUeMoD3m3ykJHwVMj6tj69iQKgX3Mz, address:77.90.31.19:4862): protocol: rpc: dial tcp 77.90.31.19:4862: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded
Aug 19 04:29:13.506689415 (251h2m54.237957045s) [noise=true , address=77.90.31.18:4835     , piece_id=W6FEWYN4QCZC5IQFUG2YXSGKHSKKFRGETXK76LFKAVACLA37H7PA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:12hAQp2Zr7WSaQSTCZf5pwSyuBHHSrZWq5EVz5hFvSA5ddWebqg, address:77.90.31.18:4835): protocol: write tcp 5.161.89.252:27410->77.90.31.18:4835: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506740311 (251h2m54.23800793s) [noise=true , address=45.91.116.7:4037     , piece_id=B42RTCUIHRS5OUFJ5S5E6QWRKEN5H34IMBURUUSSUL53Y2FUYKZQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1iNPwE2XSfXcaXAXcrH2jcyNGYeDiiWvzAcDQffrmjPwTVqVmK, address:45.91.116.7:4037): protocol: rpc: dial tcp 45.91.116.7:4037: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506774885 (251h2m54.238042535s) [noise=true , address=77.90.31.17:4801     , piece_id=A4D6GQYALBOB46EE7D2KI6422NES4M43E4DTX4CKBLNNP2VHZUJQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:127QRLNwZLcR5gnAgZrCjSUsH5T4jESmZ4kWeAnpj9YxaTSjoYe, address:77.90.31.17:4801): protocol: write tcp 5.161.89.252:48394->77.90.31.17:4801: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506808788 (251h2m54.238076419s) [noise=true , address=45.91.116.4:4952     , piece_id=ZXPM75JCQGD3XQ3BTIP4O6SOLR6XCFNBUYQKKD56RTFVCMEGFWDQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1PcGuaN6uLz1HVzMgBp4vViKJw2KrmaFatL1w9DHgiYqccMUJ9, address:45.91.116.4:4952): protocol: write tcp 5.161.89.252:61300->45.91.116.4:4952: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506825761 (251h2m54.23809341s) [noise=true , address=77.90.31.13:4645     , piece_id=TGFZGBBRZLA32ZJS3BGT7OSHJW4S34PWMYSQKCZT4RG2IUTGPZSQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg, address:77.90.31.13:4645): protocol: rpc: dial tcp 77.90.31.13:4645: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506906624 (251h2m54.238174313s) [seg_pos={0 0}, upload=261114]: Done waiting for segment. successful: 7 optimal: 14 errs: <nil>
Aug 19 04:29:13.523807608 (251h2m54.255075238s) [upload=261114]: Done uploading segments...
Aug 19 04:29:13.523824049 (251h2m54.255091679s) [upload=261114]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 7)
Duration: 567.085873ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................]
duration:
1.684040976s
FAIL (2025-08-19 04:56:53 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:34390->127.0.0.1:53: i/o timeout
Duration: 1.684063268s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
5 out of 60 is failed (threshold=3) [...................XXXX.................X...................]
duration:
530.127028ms
FAIL (2025-08-19 04:56:52 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 19 04:42:23.281867426 (251h16m4.013135076s) [upload=261352]: Starting upload
Aug 19 04:42:23.281936796 (251h16m4.013204446s) [upload=261352]: Done starting upload
Aug 19 04:42:23.282094062 (251h16m4.013361712s) [upload=261352]: Uploading segments...
Aug 19 04:42:23.282376082 (251h16m4.013643742s) [upload=261352]: Got next segment. Inline: false
Aug 19 04:42:23.282443619 (251h16m4.013711269s) [upload=261352]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:42:23.282451122 (251h16m4.013718773s) [upload=261352]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:42:23.284629308 (251h16m4.015896958s) [upload=261352]: Next segment err: metaclient: Too Many Requests
Aug 19 04:42:23.284645568 (251h16m4.015913219s) [upload=261352]: Done uploading segments...
Aug 19 04:42:23.284659365 (251h16m4.015927014s) [upload=261352]: Upload finished. err: metaclient: Too Many Requests
Duration: 530.151203ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................]
duration:
689.182972ms
target:
https://ap1.storj.io/login
FAIL (2025-08-19 04:56:52 UTC)
orbiter warning: Get "https://ap1.storj.io/login": dial tcp 35.213.185.92:443: i/o timeout
Duration: 689.204101ms core satellite ap1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)
antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................]
duration:
115.971852ms
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-08-19 04:56:47 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:41477->127.0.0.1:53: i/o timeout
Duration: 115.992962ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
6 out of 60 is failed (threshold=3) [................XXXXXX......................................]
duration:
662.352319ms
FAIL (2025-08-19 04:56:45 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
ad=261108]: Uploading piece...
Aug 19 04:27:59.298715587 (251h1m40.029983236s) [noise=true , address=45.91.116.10:4081    , piece_id=VXW7T7RYDQYFA67EV4IP5EFMNLSPHO6YMYO4NCUBCDOSPQWID6KQ, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.298767684 (251h1m40.030035364s) [noise=true , address=45.91.116.11:4119    , piece_id=6NZDOUCLI5JRQ5XXZMNLBXIVXMHR7NJY6BTCMA22QMNC3ZJJFCMA, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.299257736 (251h1m40.030525387s) [noise=true , address=45.91.116.10:4083    , piece_id=N7ABFK4IGEFBUNEAQI3SFS7IC6SCOHQD4TN2QOWDUSNRM4ZLXV7A, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.299339650 (251h1m40.030607301s) [noise=true , address=77.90.31.21:4878     , piece_id=X3FKRFUU6S3AB6ZPUNPB2HOGLJ5E7U4AWTEFH6PISRIBECP3GVSQ, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.299714204 (251h1m40.030981855s) [noise=true , address=45.91.116.8:4045     , piece_id=OH3KNS4URAIN3T4FLAPSYC2SHQKOEBLJMF2AVG7MXJULLC3P7QYA, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.299820754 (251h1m40.031088415s) [noise=true , address=77.90.31.10:4517     , piece_id=QXH4JBJTSA4VLHPS2OUAR3XF7QEL3WSX2JJT7VTZMRBVWS3OIOQQ, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.300136579 (251h1m40.031404229s) [noise=true , address=77.90.31.12:4592     , piece_id=25GKL2MHGGV6IC7JDU5MBJ4MOCCNPGMXQPYXH7A4FFFJ5EZGVZ2Q, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.300491886 (251h1m40.031759547s) [noise=true , address=45.91.116.9:4261     , piece_id=RAEAWIKMHIV4VGOZY57ZK36BNZGZ2ATXTXC4YHRABT6VWB6MGILA, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.300854618 (251h1m40.032122269s) [noise=true , address=104.36.82.2:4732     , piece_id=WKRORDL2XPRF2KRKZ55EQ2PTPJHPJSQ6NQ4D3VVFK6COGG57KDNQ, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.301277543 (251h1m40.032545194s) [noise=true , address=45.91.116.14:4190    , piece_id=SGZNGKGYEKPU4GXCULPKYIOONZZXZT6UTO7MDWYOJRJJW4WSG3VA, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.301655164 (251h1m40.032922814s) [noise=true , address=104.36.82.7:4910     , piece_id=R6GR56GOKT2K3AX75OL3BWODTZ4FJY27MIOQWFYJVY4UGR66DIBQ, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.302072699 (251h1m40.033340349s) [noise=true , address=104.36.82.16:4294    , piece_id=GT3KQOCWD2YTWBQDAO5QJKYIMZINUF4MESKP6F3P6J73R4IY3NDQ, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:28:02.354368898 (251h1m43.085636557s) [noise=true , address=77.90.31.21:4878     , piece_id=X3FKRFUU6S3AB6ZPUNPB2HOGLJ5E7U4AWTEFH6PISRIBECP3GVSQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: write tcp 5.161.89.252:64042->77.90.31.21:4878: write: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:02.385567243 (251h1m43.116834893s) [noise=true , address=77.90.31.12:4592     , piece_id=25GKL2MHGGV6IC7JDU5MBJ4MOCCNPGMXQPYXH7A4FFFJ5EZGVZ2Q, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 5.161.89.252:34306->77.90.31.12:4592: write: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op
Aug 19 04:28:29.225676153 (251h2m9.956943773s) [upload=261108]: Next segment err: context deadline exceeded
Aug 19 04:28:29.225998790 (251h2m9.95726642s) [noise=true , address=104.36.82.2:4732     , piece_id=WKRORDL2XPRF2KRKZ55EQ2PTPJHPJSQ6NQ4D3VVFK6COGG57KDNQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226039707 (251h2m9.957307326s) [noise=true , address=45.91.116.11:4119    , piece_id=6NZDOUCLI5JRQ5XXZMNLBXIVXMHR7NJY6BTCMA22QMNC3ZJJFCMA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226072177 (251h2m9.957339798s) [noise=true , address=45.91.116.8:4045     , piece_id=OH3KNS4URAIN3T4FLAPSYC2SHQKOEBLJMF2AVG7MXJULLC3P7QYA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226223943 (251h2m9.957491573s) [noise=true , address=45.91.116.14:4190    , piece_id=SGZNGKGYEKPU4GXCULPKYIOONZZXZT6UTO7MDWYOJRJJW4WSG3VA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226274027 (251h2m9.957541647s) [noise=true , address=45.91.116.13:4154    , piece_id=WM56E23Z7FLRF33HYDXONXHODJDHPCJFWGVC3NI7RUTZUESWO4MQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226310786 (251h2m9.957578416s) [noise=true , address=77.90.31.17:4771     , piece_id=3BVZK3ZL7N5FC3OJGFFYTVEG7INVMLMG2QBPXX3UC3EJVXHBGW3Q, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226348096 (251h2m9.957615726s) [noise=true , address=104.36.82.11:4114    , piece_id=P4RJRUZPRTFPRK5WWHKUVEKJKXGPPPCIV4RPFIZOBO2CPOXSTSIQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: rpc: dial tcp 104.36.82.11:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226392119 (251h2m9.957659749s) [noise=true , address=45.91.116.10:4081    , piece_id=VXW7T7RYDQYFA67EV4IP5EFMNLSPHO6YMYO4NCUBCDOSPQWID6KQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 5.161.89.252:44604->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226421564 (251h2m9.957689195s) [noise=true , address=45.91.116.9:4261     , piece_id=RAEAWIKMHIV4VGOZY57ZK36BNZGZ2ATXTXC4YHRABT6VWB6MGILA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226443746 (251h2m9.957711377s) [noise=true , address=45.91.116.4:4940     , piece_id=GYF7CH3463KOWU7DE7WKLYTXYXLTRMPOIK4K3D64ANKOERN33FGQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 5.161.89.252:53922->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226474684 (251h2m9.957742315s) [noise=true , address=104.36.82.7:4910     , piece_id=R6GR56GOKT2K3AX75OL3BWODTZ4FJY27MIOQWFYJVY4UGR66DIBQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226493810 (251h2m9.95776143s) [noise=true , address=45.91.116.10:4083    , piece_id=N7ABFK4IGEFBUNEAQI3SFS7IC6SCOHQD4TN2QOWDUSNRM4ZLXV7A, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: rpc: dial tcp 45.91.116.10:4083: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226509199 (251h2m9.957776829s) [noise=true , address=104.36.82.16:4294    , piece_id=GT3KQOCWD2YTWBQDAO5QJKYIMZINUF4MESKP6F3P6J73R4IY3NDQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:104.36.82.16:4294): protocol: write tcp 5.161.89.252:25886->104.36.82.16:4294: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226527894 (251h2m9.957795524s) [noise=true , address=77.90.31.10:4517     , piece_id=QXH4JBJTSA4VLHPS2OUAR3XF7QEL3WSX2JJT7VTZMRBVWS3OIOQQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226544845 (251h2m9.957812476s) [seg_pos={0 0}, upload=261108]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 662.372488ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................]
duration:
4.714560841s
FAIL (2025-08-19 04:56:44 UTC)
orbiter warning: operation: StatBucket
uplink: bucket: metaclient: context deadline exceeded
Duration: 4.714581961s core uplink ap1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)
antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................]
duration:
39.00943ms
FAIL (2025-08-19 04:56:42 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:26171->127.0.0.1:53: i/o timeout
Duration: 39.036741ms edge authservice http us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................]
duration:
430.870179ms
FAIL (2025-08-19 04:56:41 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:26171->127.0.0.1:53: i/o timeout
Duration: 431.072088ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................]
duration:
489.532919ms
FAIL (2025-08-19 04:56:35 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:26171->127.0.0.1:53: i/o timeout
Duration: 489.555881ms edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)
antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................]
duration:
114.22412ms
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-08-19 04:56:31 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:41477->127.0.0.1:53: i/o timeout
Duration: 114.243377ms edge gateway vhost bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)
antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................]
duration:
573.802672ms
FAIL (2025-08-19 04:56:28 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:34390->127.0.0.1:53: i/o timeout
Duration: 573.848448ms edge authservice http bgp eu1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)
antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................]
duration:
208.142691ms
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318
FAIL (2025-08-19 04:56:21 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:41477->127.0.0.1:53: i/o timeout
Duration: 208.192505ms edge gateway vhost bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
5 out of 60 is failed (threshold=3) [....................XXXX.................X..................]
duration:
967.329683ms
FAIL (2025-08-19 04:56:15 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 19 04:42:23.281867426 (251h16m4.013135076s) [upload=261352]: Starting upload
Aug 19 04:42:23.281936796 (251h16m4.013204446s) [upload=261352]: Done starting upload
Aug 19 04:42:23.282094062 (251h16m4.013361712s) [upload=261352]: Uploading segments...
Aug 19 04:42:23.282376082 (251h16m4.013643742s) [upload=261352]: Got next segment. Inline: false
Aug 19 04:42:23.282443619 (251h16m4.013711269s) [upload=261352]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 04:42:23.282451122 (251h16m4.013718773s) [upload=261352]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 04:42:23.284629308 (251h16m4.015896958s) [upload=261352]: Next segment err: metaclient: Too Many Requests
Aug 19 04:42:23.284645568 (251h16m4.015913219s) [upload=261352]: Done uploading segments...
Aug 19 04:42:23.284659365 (251h16m4.015927014s) [upload=261352]: Upload finished. err: metaclient: Too Many Requests
Duration: 967.353508ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing)
antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................]
duration:
9.617937ms
target:
http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt
FAIL (2025-08-19 04:56:15 UTC)
orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp: lookup link.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:34965->127.0.0.1:53: i/o timeout
Duration: 9.637845ms edge linksharing us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
6 out of 60 is failed (threshold=3) [.................XXXXXX.....................................]
duration:
553.652215ms
FAIL (2025-08-19 04:56:13 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
ad=261108]: Uploading piece...
Aug 19 04:27:59.298715587 (251h1m40.029983236s) [noise=true , address=45.91.116.10:4081    , piece_id=VXW7T7RYDQYFA67EV4IP5EFMNLSPHO6YMYO4NCUBCDOSPQWID6KQ, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.298767684 (251h1m40.030035364s) [noise=true , address=45.91.116.11:4119    , piece_id=6NZDOUCLI5JRQ5XXZMNLBXIVXMHR7NJY6BTCMA22QMNC3ZJJFCMA, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.299257736 (251h1m40.030525387s) [noise=true , address=45.91.116.10:4083    , piece_id=N7ABFK4IGEFBUNEAQI3SFS7IC6SCOHQD4TN2QOWDUSNRM4ZLXV7A, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.299339650 (251h1m40.030607301s) [noise=true , address=77.90.31.21:4878     , piece_id=X3FKRFUU6S3AB6ZPUNPB2HOGLJ5E7U4AWTEFH6PISRIBECP3GVSQ, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.299714204 (251h1m40.030981855s) [noise=true , address=45.91.116.8:4045     , piece_id=OH3KNS4URAIN3T4FLAPSYC2SHQKOEBLJMF2AVG7MXJULLC3P7QYA, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.299820754 (251h1m40.031088415s) [noise=true , address=77.90.31.10:4517     , piece_id=QXH4JBJTSA4VLHPS2OUAR3XF7QEL3WSX2JJT7VTZMRBVWS3OIOQQ, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.300136579 (251h1m40.031404229s) [noise=true , address=77.90.31.12:4592     , piece_id=25GKL2MHGGV6IC7JDU5MBJ4MOCCNPGMXQPYXH7A4FFFJ5EZGVZ2Q, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.300491886 (251h1m40.031759547s) [noise=true , address=45.91.116.9:4261     , piece_id=RAEAWIKMHIV4VGOZY57ZK36BNZGZ2ATXTXC4YHRABT6VWB6MGILA, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.300854618 (251h1m40.032122269s) [noise=true , address=104.36.82.2:4732     , piece_id=WKRORDL2XPRF2KRKZ55EQ2PTPJHPJSQ6NQ4D3VVFK6COGG57KDNQ, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.301277543 (251h1m40.032545194s) [noise=true , address=45.91.116.14:4190    , piece_id=SGZNGKGYEKPU4GXCULPKYIOONZZXZT6UTO7MDWYOJRJJW4WSG3VA, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.301655164 (251h1m40.032922814s) [noise=true , address=104.36.82.7:4910     , piece_id=R6GR56GOKT2K3AX75OL3BWODTZ4FJY27MIOQWFYJVY4UGR66DIBQ, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:27:59.302072699 (251h1m40.033340349s) [noise=true , address=104.36.82.16:4294    , piece_id=GT3KQOCWD2YTWBQDAO5QJKYIMZINUF4MESKP6F3P6J73R4IY3NDQ, seg_pos={0 0}, upload=261108]: Uploading piece...
Aug 19 04:28:02.354368898 (251h1m43.085636557s) [noise=true , address=77.90.31.21:4878     , piece_id=X3FKRFUU6S3AB6ZPUNPB2HOGLJ5E7U4AWTEFH6PISRIBECP3GVSQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: write tcp 5.161.89.252:64042->77.90.31.21:4878: write: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:02.385567243 (251h1m43.116834893s) [noise=true , address=77.90.31.12:4592     , piece_id=25GKL2MHGGV6IC7JDU5MBJ4MOCCNPGMXQPYXH7A4FFFJ5EZGVZ2Q, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 5.161.89.252:34306->77.90.31.12:4592: write: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op
Aug 19 04:28:29.225676153 (251h2m9.956943773s) [upload=261108]: Next segment err: context deadline exceeded
Aug 19 04:28:29.225998790 (251h2m9.95726642s) [noise=true , address=104.36.82.2:4732     , piece_id=WKRORDL2XPRF2KRKZ55EQ2PTPJHPJSQ6NQ4D3VVFK6COGG57KDNQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226039707 (251h2m9.957307326s) [noise=true , address=45.91.116.11:4119    , piece_id=6NZDOUCLI5JRQ5XXZMNLBXIVXMHR7NJY6BTCMA22QMNC3ZJJFCMA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226072177 (251h2m9.957339798s) [noise=true , address=45.91.116.8:4045     , piece_id=OH3KNS4URAIN3T4FLAPSYC2SHQKOEBLJMF2AVG7MXJULLC3P7QYA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226223943 (251h2m9.957491573s) [noise=true , address=45.91.116.14:4190    , piece_id=SGZNGKGYEKPU4GXCULPKYIOONZZXZT6UTO7MDWYOJRJJW4WSG3VA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226274027 (251h2m9.957541647s) [noise=true , address=45.91.116.13:4154    , piece_id=WM56E23Z7FLRF33HYDXONXHODJDHPCJFWGVC3NI7RUTZUESWO4MQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226310786 (251h2m9.957578416s) [noise=true , address=77.90.31.17:4771     , piece_id=3BVZK3ZL7N5FC3OJGFFYTVEG7INVMLMG2QBPXX3UC3EJVXHBGW3Q, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226348096 (251h2m9.957615726s) [noise=true , address=104.36.82.11:4114    , piece_id=P4RJRUZPRTFPRK5WWHKUVEKJKXGPPPCIV4RPFIZOBO2CPOXSTSIQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: rpc: dial tcp 104.36.82.11:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226392119 (251h2m9.957659749s) [noise=true , address=45.91.116.10:4081    , piece_id=VXW7T7RYDQYFA67EV4IP5EFMNLSPHO6YMYO4NCUBCDOSPQWID6KQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 5.161.89.252:44604->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226421564 (251h2m9.957689195s) [noise=true , address=45.91.116.9:4261     , piece_id=RAEAWIKMHIV4VGOZY57ZK36BNZGZ2ATXTXC4YHRABT6VWB6MGILA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226443746 (251h2m9.957711377s) [noise=true , address=45.91.116.4:4940     , piece_id=GYF7CH3463KOWU7DE7WKLYTXYXLTRMPOIK4K3D64ANKOERN33FGQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 5.161.89.252:53922->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226474684 (251h2m9.957742315s) [noise=true , address=104.36.82.7:4910     , piece_id=R6GR56GOKT2K3AX75OL3BWODTZ4FJY27MIOQWFYJVY4UGR66DIBQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226493810 (251h2m9.95776143s) [noise=true , address=45.91.116.10:4083    , piece_id=N7ABFK4IGEFBUNEAQI3SFS7IC6SCOHQD4TN2QOWDUSNRM4ZLXV7A, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: rpc: dial tcp 45.91.116.10:4083: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226509199 (251h2m9.957776829s) [noise=true , address=104.36.82.16:4294    , piece_id=GT3KQOCWD2YTWBQDAO5QJKYIMZINUF4MESKP6F3P6J73R4IY3NDQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:104.36.82.16:4294): protocol: write tcp 5.161.89.252:25886->104.36.82.16:4294: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226527894 (251h2m9.957795524s) [noise=true , address=77.90.31.10:4517     , piece_id=QXH4JBJTSA4VLHPS2OUAR3XF7QEL3WSX2JJT7VTZMRBVWS3OIOQQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:29.226544845 (251h2m9.957812476s) [seg_pos={0 0}, upload=261108]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 553.67607ms core uplink us-select-1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)
antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................]
duration:
529.974455ms
FAIL (2025-08-19 04:56:11 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:34390->127.0.0.1:53: i/o timeout
Duration: 529.995143ms edge authservice http bgp eu1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1)
antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................]
duration:
877.009098ms
FAIL (2025-08-19 04:56:08 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:34390->127.0.0.1:53: i/o timeout
Duration: 877.030368ms edge authservice http bgp ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................]
duration:
221.291286ms
FAIL (2025-08-19 04:55:59 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:26171->127.0.0.1:53: i/o timeout
Duration: 221.313358ms edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)
antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................]
duration:
32.269082ms
FAIL (2025-08-19 04:55:56 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:26171->127.0.0.1:53: i/o timeout
Duration: 32.292506ms edge authservice http us-select-1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)
antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................]
duration:
552.682653ms
FAIL (2025-08-19 04:55:50 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:34390->127.0.0.1:53: i/o timeout
Duration: 552.703431ms edge authservice http bgp eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................]
duration:
1.369573744s
FAIL (2025-08-19 04:55:49 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:34390->127.0.0.1:53: i/o timeout
Duration: 1.369591177s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................]
duration:
691.709519ms
target:
https://ap1.storj.io/login
FAIL (2025-08-19 04:55:48 UTC)
orbiter warning: Get "https://ap1.storj.io/login": dial tcp 35.213.185.92:443: i/o timeout
Duration: 691.729144ms core satellite ap1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................]
duration:
944.413798ms
FAIL (2025-08-19 04:55:47 UTC)
orbiter warning: operation: StatBucket
uplink: bucket: metaclient: context deadline exceeded
Duration: 944.432172ms core uplink eu1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)
antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................]
duration:
254.664436ms
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318
FAIL (2025-08-19 04:55:43 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:41477->127.0.0.1:53: i/o timeout
Duration: 254.688231ms edge gateway vhost bgp eu1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1)
antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................]
duration:
958.959605ms
FAIL (2025-08-19 04:55:42 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:34390->127.0.0.1:53: i/o timeout
Duration: 958.977629ms edge authservice http bgp ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
4 out of 60 is failed (threshold=3) [...................XXXX.....................................]
duration:
554.457146ms
FAIL (2025-08-19 04:55:39 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 7)
======LOGS======
g piece...
Aug 19 04:28:13.563886368 (251h1m54.295154018s) [noise=true , address=104.36.82.5:4843     , piece_id=Q3SLO2FDTYYYYOQZARVS3SHAPM5J3GCLNFQ355QAV7U5MWKMZQQA, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.564531581 (251h1m54.295799231s) [noise=true , address=45.91.116.7:4037     , piece_id=B42RTCUIHRS5OUFJ5S5E6QWRKEN5H34IMBURUUSSUL53Y2FUYKZQ, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.564716169 (251h1m54.295983828s) [noise=true , address=104.36.82.15:4289    , piece_id=GVBKZHWTPFDGYCM6U5EUYLBOTGCTTLPDWW2WZKZZEVNG4UZ3XEYA, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.565358756 (251h1m54.296626407s) [noise=true , address=77.90.31.21:4891     , piece_id=G57I4V2AOHDBI77GIIULZXTGCFC2BP6CQRX25KA7UNNIJ3WTAGPQ, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.565475757 (251h1m54.296743417s) [noise=true , address=45.91.116.9:4286     , piece_id=KUMIYTCCEU7XS2NMUTPXDSD7TMNASOBWGNWNDD7L6DMEAPXVIFWA, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.566262315 (251h1m54.297529975s) [noise=true , address=104.36.82.44:4880    , piece_id=ZRDRV5AWRCYSUZJANVESK3XUCMLQQXF33WPV6QPX6HAZBYHXOUUQ, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.566506594 (251h1m54.297774285s) [noise=true , address=45.91.116.12:4243    , piece_id=GNLESUJ3FCKKUC7HOR7Z532H7LQT6ZOTOKLBGRN2R6UCJL4S3VFQ, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.567576856 (251h1m54.298844507s) [noise=true , address=45.91.116.4:4952     , piece_id=ZXPM75JCQGD3XQ3BTIP4O6SOLR6XCFNBUYQKKD56RTFVCMEGFWDQ, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.575982240 (251h1m54.30724989s) [noise=true , address=77.90.31.19:4862     , piece_id=7CQOCX2FYKPQCTCBFMN7BMTD3FE2D2X4YIHFLCL22AOSYLKUO33A, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.576556950 (251h1m54.3078246s) [noise=true , address=77.90.31.17:4801     , piece_id=A4D6GQYALBOB46EE7D2KI6422NES4M43E4DTX4CKBLNNP2VHZUJQ, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.577052442 (251h1m54.308320102s) [noise=true , address=104.36.82.13:4204    , piece_id=LWFS4W2N7K6SWYKXYHZRJACAYYDBIB7OGVETZWKVTWWWJBZPS33A, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.577571397 (251h1m54.308839058s) [noise=true , address=104.36.82.12:4156    , piece_id=RMBTQYXVS5NKE2ON3WW5T2CBE6S7S6T74VVVLMUSM5C44UXGRAYA, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.581420654 (251h1m54.312688304s) [noise=true , address=77.90.31.12:4596     , piece_id=AEOFPZ64F5FUTWHLMKFVKQGNGYJYOIU7OZPLYGS7ULR52F5XSNYQ, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.582252959 (251h1m54.313520609s) [noise=true , address=77.90.31.18:4835     , piece_id=W6FEWYN4QCZC5IQFUG2YXSGKHSKKFRGETXK76LFKAVACLA37H7PA, seg_pos={0 0}, upload=261114]: Uploading piece...
Aug 19 04:28:13.672688665 (251h1m54.403956365s) [upload=261114]: Next returned nil segment
Aug 19 04:28:13.672710426 (251h1m54.403978086s) [upload=261114]: Waiting for error group managing segments...
Aug 19 04:28:13.681638350 (251h1m54.412906s) [noise=true , address=79.127.219.48:4085   , piece_id=AT2YETVN3DCMAPBL45XWQQCTW5FR6JC4PWN3CRIMZPQW5OG3RKKQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil>
Aug 19 04:28:28.955286140 (251h2m9.68655377s) [noise=true , address=77.90.31.12:4596     , piece_id=AEOFPZ64F5FUTWHLMKFVKQGNGYJYOIU7OZPLYGS7ULR52F5XSNYQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1xgzDjjKfA3YT4tjGq6Esw9xfhA4xWMWnDbyAmNFUcP85qyFGf, address:77.90.31.12:4596): protocol: rpc: dial tcp 77.90.31.12:4596: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:28:45.285087265 (251h2m26.016354885s) [noise=true , address=104.36.82.60:4354    , piece_id=76HWEIKV5672IF34IRITARBMKGTN7OJU6GHQZZCDAP2ZUR2JNJFA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil>
Aug 19 04:28:45.286367592 (251h2m26.017635222s) [noise=true , address=104.36.82.13:4204    , piece_id=LWFS4W2N7K6SWYKXYHZRJACAYYDBIB7OGVETZWKVTWWWJBZPS33A, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil>
Aug 19 04:28:45.286429859 (251h2m26.017697489s) [noise=true , address=104.36.82.12:4156    , piece_id=RMBTQYXVS5NKE2ON3WW5T2CBE6S7S6T74VVVLMUSM5C44UXGRAYA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil>
Aug 19 04:28:45.288231957 (251h2m26.019499578s) [noise=true , address=104.36.82.15:4289    , piece_id=GVBKZHWTPFDGYCM6U5EUYLBOTGCTTLPDWW2WZKZZEVNG4UZ3XEYA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil>
Aug 19 04:28:45.316902561 (251h2m26.048170191s) [noise=true , address=104.36.82.44:4880    , piece_id=ZRDRV5AWRCYSUZJANVESK3XUCMLQQXF33WPV6QPX6HAZBYHXOUUQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil>
Aug 19 04:28:46.178003461 (251h2m26.90927112s) [noise=true , address=104.36.82.5:4843     , piece_id=Q3SLO2FDTYYYYOQZARVS3SHAPM5J3GCLNFQ355QAV7U5MWKMZQQA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil>
Aug 19 04:29:13.506175578 (251h2m54.237443199s) [noise=true , address=45.91.116.6:4988     , piece_id=X36K53BYKX53WMZKQWN2FFKCKTHNLIY3KLLPXMQN2UBKZM2CTT6Q, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1TrcoCHhAJeEykP9hJNpygoPSdgFV5v6GzL48JpyNPiAj5ZJRS, address:45.91.116.6:4988): protocol: rpc: dial tcp 45.91.116.6:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506222607 (251h2m54.237490227s) [noise=true , address=45.91.116.9:4286     , piece_id=KUMIYTCCEU7XS2NMUTPXDSD7TMNASOBWGNWNDD7L6DMEAPXVIFWA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:19DfbHig9tzF4VRHtSWkExwUS2bdZ8GFgKnAD9wvMBG7qaesNa, address:45.91.116.9:4286): protocol: rpc: dial tcp 45.91.116.9:4286: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506254087 (251h2m54.237521717s) [noise=true , address=45.91.116.12:4243    , piece_id=GNLESUJ3FCKKUC7HOR7Z532H7LQT6ZOTOKLBGRN2R6UCJL4S3VFQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1e9VEMmhYq6PxqvTpxsoUHzqYVwujfj1UG7HSZFeiu8CderMpM, address:45.91.116.12:4243): protocol: rpc: dial tcp 45.91.116.12:4243: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506625304 (251h2m54.237892934s) [noise=true , address=77.90.31.21:4891     , piece_id=G57I4V2AOHDBI77GIIULZXTGCFC2BP6CQRX25KA7UNNIJ3WTAGPQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:17RDAerfx8AW5NsgWLTXLNDbuAEWQFT3SVM4iwdmuBzWrML83g, address:77.90.31.21:4891): protocol: rpc: dial tcp 77.90.31.21:4891: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506656563 (251h2m54.237924213s) [noise=true , address=77.90.31.19:4862     , piece_id=7CQOCX2FYKPQCTCBFMN7BMTD3FE2D2X4YIHFLCL22AOSYLKUO33A, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1Tvmo3nSgBPcFYW9MmZQGUeMoD3m3ykJHwVMj6tj69iQKgX3Mz, address:77.90.31.19:4862): protocol: rpc: dial tcp 77.90.31.19:4862: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded
Aug 19 04:29:13.506689415 (251h2m54.237957045s) [noise=true , address=77.90.31.18:4835     , piece_id=W6FEWYN4QCZC5IQFUG2YXSGKHSKKFRGETXK76LFKAVACLA37H7PA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:12hAQp2Zr7WSaQSTCZf5pwSyuBHHSrZWq5EVz5hFvSA5ddWebqg, address:77.90.31.18:4835): protocol: write tcp 5.161.89.252:27410->77.90.31.18:4835: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506740311 (251h2m54.23800793s) [noise=true , address=45.91.116.7:4037     , piece_id=B42RTCUIHRS5OUFJ5S5E6QWRKEN5H34IMBURUUSSUL53Y2FUYKZQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1iNPwE2XSfXcaXAXcrH2jcyNGYeDiiWvzAcDQffrmjPwTVqVmK, address:45.91.116.7:4037): protocol: rpc: dial tcp 45.91.116.7:4037: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506774885 (251h2m54.238042535s) [noise=true , address=77.90.31.17:4801     , piece_id=A4D6GQYALBOB46EE7D2KI6422NES4M43E4DTX4CKBLNNP2VHZUJQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:127QRLNwZLcR5gnAgZrCjSUsH5T4jESmZ4kWeAnpj9YxaTSjoYe, address:77.90.31.17:4801): protocol: write tcp 5.161.89.252:48394->77.90.31.17:4801: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506808788 (251h2m54.238076419s) [noise=true , address=45.91.116.4:4952     , piece_id=ZXPM75JCQGD3XQ3BTIP4O6SOLR6XCFNBUYQKKD56RTFVCMEGFWDQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1PcGuaN6uLz1HVzMgBp4vViKJw2KrmaFatL1w9DHgiYqccMUJ9, address:45.91.116.4:4952): protocol: write tcp 5.161.89.252:61300->45.91.116.4:4952: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506825761 (251h2m54.23809341s) [noise=true , address=77.90.31.13:4645     , piece_id=TGFZGBBRZLA32ZJS3BGT7OSHJW4S34PWMYSQKCZT4RG2IUTGPZSQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg, address:77.90.31.13:4645): protocol: rpc: dial tcp 77.90.31.13:4645: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 19 04:29:13.506906624 (251h2m54.238174313s) [seg_pos={0 0}, upload=261114]: Done waiting for segment. successful: 7 optimal: 14 errs: <nil>
Aug 19 04:29:13.523807608 (251h2m54.255075238s) [upload=261114]: Done uploading segments...
Aug 19 04:29:13.523824049 (251h2m54.255091679s) [upload=261114]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 7)
Duration: 554.476663ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................]
duration:
374.997865ms
FAIL (2025-08-19 04:55:38 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:26171->127.0.0.1:53: i/o timeout
Duration: 375.29764ms edge gateway us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................]
duration:
4.666576021s
FAIL (2025-08-19 04:55:35 UTC)
orbiter warning: operation: StatBucket
uplink: bucket: metaclient: context deadline exceeded
Duration: 4.666729651s core uplink ap1