Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...............XXXX.................X.......................] duration: 531.284532ms FAIL (2025-08-19 05:00:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:42:23.281867426 (251h16m4.013135076s) [upload=261352]: Starting upload Aug 19 04:42:23.281936796 (251h16m4.013204446s) [upload=261352]: Done starting upload Aug 19 04:42:23.282094062 (251h16m4.013361712s) [upload=261352]: Uploading segments... Aug 19 04:42:23.282376082 (251h16m4.013643742s) [upload=261352]: Got next segment. Inline: false Aug 19 04:42:23.282443619 (251h16m4.013711269s) [upload=261352]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 04:42:23.282451122 (251h16m4.013718773s) [upload=261352]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 04:42:23.284629308 (251h16m4.015896958s) [upload=261352]: Next segment err: metaclient: Too Many Requests Aug 19 04:42:23.284645568 (251h16m4.015913219s) [upload=261352]: Done uploading segments... Aug 19 04:42:23.284659365 (251h16m4.015927014s) [upload=261352]: Upload finished. err: metaclient: Too Many Requests Duration: 531.433774ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...........XXXXXX...........................................] duration: 1.595031444s FAIL (2025-08-19 05:00:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ad=261108]: Uploading piece... Aug 19 04:27:59.298715587 (251h1m40.029983236s) [noise=true , address=45.91.116.10:4081 , piece_id=VXW7T7RYDQYFA67EV4IP5EFMNLSPHO6YMYO4NCUBCDOSPQWID6KQ, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.298767684 (251h1m40.030035364s) [noise=true , address=45.91.116.11:4119 , piece_id=6NZDOUCLI5JRQ5XXZMNLBXIVXMHR7NJY6BTCMA22QMNC3ZJJFCMA, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.299257736 (251h1m40.030525387s) [noise=true , address=45.91.116.10:4083 , piece_id=N7ABFK4IGEFBUNEAQI3SFS7IC6SCOHQD4TN2QOWDUSNRM4ZLXV7A, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.299339650 (251h1m40.030607301s) [noise=true , address=77.90.31.21:4878 , piece_id=X3FKRFUU6S3AB6ZPUNPB2HOGLJ5E7U4AWTEFH6PISRIBECP3GVSQ, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.299714204 (251h1m40.030981855s) [noise=true , address=45.91.116.8:4045 , piece_id=OH3KNS4URAIN3T4FLAPSYC2SHQKOEBLJMF2AVG7MXJULLC3P7QYA, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.299820754 (251h1m40.031088415s) [noise=true , address=77.90.31.10:4517 , piece_id=QXH4JBJTSA4VLHPS2OUAR3XF7QEL3WSX2JJT7VTZMRBVWS3OIOQQ, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.300136579 (251h1m40.031404229s) [noise=true , address=77.90.31.12:4592 , piece_id=25GKL2MHGGV6IC7JDU5MBJ4MOCCNPGMXQPYXH7A4FFFJ5EZGVZ2Q, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.300491886 (251h1m40.031759547s) [noise=true , address=45.91.116.9:4261 , piece_id=RAEAWIKMHIV4VGOZY57ZK36BNZGZ2ATXTXC4YHRABT6VWB6MGILA, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.300854618 (251h1m40.032122269s) [noise=true , address=104.36.82.2:4732 , piece_id=WKRORDL2XPRF2KRKZ55EQ2PTPJHPJSQ6NQ4D3VVFK6COGG57KDNQ, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.301277543 (251h1m40.032545194s) [noise=true , address=45.91.116.14:4190 , piece_id=SGZNGKGYEKPU4GXCULPKYIOONZZXZT6UTO7MDWYOJRJJW4WSG3VA, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.301655164 (251h1m40.032922814s) [noise=true , address=104.36.82.7:4910 , piece_id=R6GR56GOKT2K3AX75OL3BWODTZ4FJY27MIOQWFYJVY4UGR66DIBQ, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.302072699 (251h1m40.033340349s) [noise=true , address=104.36.82.16:4294 , piece_id=GT3KQOCWD2YTWBQDAO5QJKYIMZINUF4MESKP6F3P6J73R4IY3NDQ, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:28:02.354368898 (251h1m43.085636557s) [noise=true , address=77.90.31.21:4878 , piece_id=X3FKRFUU6S3AB6ZPUNPB2HOGLJ5E7U4AWTEFH6PISRIBECP3GVSQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: write tcp 5.161.89.252:64042->77.90.31.21:4878: write: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:02.385567243 (251h1m43.116834893s) [noise=true , address=77.90.31.12:4592 , piece_id=25GKL2MHGGV6IC7JDU5MBJ4MOCCNPGMXQPYXH7A4FFFJ5EZGVZ2Q, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 5.161.89.252:34306->77.90.31.12:4592: write: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Aug 19 04:28:29.225676153 (251h2m9.956943773s) [upload=261108]: Next segment err: context deadline exceeded Aug 19 04:28:29.225998790 (251h2m9.95726642s) [noise=true , address=104.36.82.2:4732 , piece_id=WKRORDL2XPRF2KRKZ55EQ2PTPJHPJSQ6NQ4D3VVFK6COGG57KDNQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226039707 (251h2m9.957307326s) [noise=true , address=45.91.116.11:4119 , piece_id=6NZDOUCLI5JRQ5XXZMNLBXIVXMHR7NJY6BTCMA22QMNC3ZJJFCMA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226072177 (251h2m9.957339798s) [noise=true , address=45.91.116.8:4045 , piece_id=OH3KNS4URAIN3T4FLAPSYC2SHQKOEBLJMF2AVG7MXJULLC3P7QYA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226223943 (251h2m9.957491573s) [noise=true , address=45.91.116.14:4190 , piece_id=SGZNGKGYEKPU4GXCULPKYIOONZZXZT6UTO7MDWYOJRJJW4WSG3VA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226274027 (251h2m9.957541647s) [noise=true , address=45.91.116.13:4154 , piece_id=WM56E23Z7FLRF33HYDXONXHODJDHPCJFWGVC3NI7RUTZUESWO4MQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226310786 (251h2m9.957578416s) [noise=true , address=77.90.31.17:4771 , piece_id=3BVZK3ZL7N5FC3OJGFFYTVEG7INVMLMG2QBPXX3UC3EJVXHBGW3Q, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226348096 (251h2m9.957615726s) [noise=true , address=104.36.82.11:4114 , piece_id=P4RJRUZPRTFPRK5WWHKUVEKJKXGPPPCIV4RPFIZOBO2CPOXSTSIQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: rpc: dial tcp 104.36.82.11:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226392119 (251h2m9.957659749s) [noise=true , address=45.91.116.10:4081 , piece_id=VXW7T7RYDQYFA67EV4IP5EFMNLSPHO6YMYO4NCUBCDOSPQWID6KQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 5.161.89.252:44604->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226421564 (251h2m9.957689195s) [noise=true , address=45.91.116.9:4261 , piece_id=RAEAWIKMHIV4VGOZY57ZK36BNZGZ2ATXTXC4YHRABT6VWB6MGILA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226443746 (251h2m9.957711377s) [noise=true , address=45.91.116.4:4940 , piece_id=GYF7CH3463KOWU7DE7WKLYTXYXLTRMPOIK4K3D64ANKOERN33FGQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 5.161.89.252:53922->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226474684 (251h2m9.957742315s) [noise=true , address=104.36.82.7:4910 , piece_id=R6GR56GOKT2K3AX75OL3BWODTZ4FJY27MIOQWFYJVY4UGR66DIBQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226493810 (251h2m9.95776143s) [noise=true , address=45.91.116.10:4083 , piece_id=N7ABFK4IGEFBUNEAQI3SFS7IC6SCOHQD4TN2QOWDUSNRM4ZLXV7A, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: rpc: dial tcp 45.91.116.10:4083: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226509199 (251h2m9.957776829s) [noise=true , address=104.36.82.16:4294 , piece_id=GT3KQOCWD2YTWBQDAO5QJKYIMZINUF4MESKP6F3P6J73R4IY3NDQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:104.36.82.16:4294): protocol: write tcp 5.161.89.252:25886->104.36.82.16:4294: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226527894 (251h2m9.957795524s) [noise=true , address=77.90.31.10:4517 , piece_id=QXH4JBJTSA4VLHPS2OUAR3XF7QEL3WSX2JJT7VTZMRBVWS3OIOQQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226544845 (251h2m9.957812476s) [seg_pos={0 0}, upload=261108]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 1.59506136s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 510.82517ms FAIL (2025-08-19 05:00:40 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:26171->127.0.0.1:53: i/o timeout Duration: 510.837263ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 1.980549983s FAIL (2025-08-19 05:00:32 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:34390->127.0.0.1:53: i/o timeout Duration: 1.980560844s core uplink us-select-1 us1-placement-20
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 923.212999ms FAIL (2025-08-19 05:00:32 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:34390->127.0.0.1:53: i/o timeout Duration: 923.233047ms edge authservice http bgp ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 4.630863037s FAIL (2025-08-19 05:00:28 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 4.630883505s core uplink ap1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 140.89005ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-19 05:00:14 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:41477->127.0.0.1:53: i/o timeout Duration: 140.921719ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 685.59671ms target: https://ap1.storj.io/login FAIL (2025-08-19 05:00:13 UTC) orbiter warning: Get "https://ap1.storj.io/login": dial tcp 35.213.185.92:443: i/o timeout Duration: 685.613271ms core satellite ap1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 575.616517ms FAIL (2025-08-19 05:00:12 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:34390->127.0.0.1:53: i/o timeout Duration: 575.636334ms edge authservice http bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............XXXX..........................................] duration: 1.312926142s FAIL (2025-08-19 05:00:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 7) ======LOGS====== g piece... Aug 19 04:28:13.563886368 (251h1m54.295154018s) [noise=true , address=104.36.82.5:4843 , piece_id=Q3SLO2FDTYYYYOQZARVS3SHAPM5J3GCLNFQ355QAV7U5MWKMZQQA, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.564531581 (251h1m54.295799231s) [noise=true , address=45.91.116.7:4037 , piece_id=B42RTCUIHRS5OUFJ5S5E6QWRKEN5H34IMBURUUSSUL53Y2FUYKZQ, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.564716169 (251h1m54.295983828s) [noise=true , address=104.36.82.15:4289 , piece_id=GVBKZHWTPFDGYCM6U5EUYLBOTGCTTLPDWW2WZKZZEVNG4UZ3XEYA, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.565358756 (251h1m54.296626407s) [noise=true , address=77.90.31.21:4891 , piece_id=G57I4V2AOHDBI77GIIULZXTGCFC2BP6CQRX25KA7UNNIJ3WTAGPQ, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.565475757 (251h1m54.296743417s) [noise=true , address=45.91.116.9:4286 , piece_id=KUMIYTCCEU7XS2NMUTPXDSD7TMNASOBWGNWNDD7L6DMEAPXVIFWA, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.566262315 (251h1m54.297529975s) [noise=true , address=104.36.82.44:4880 , piece_id=ZRDRV5AWRCYSUZJANVESK3XUCMLQQXF33WPV6QPX6HAZBYHXOUUQ, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.566506594 (251h1m54.297774285s) [noise=true , address=45.91.116.12:4243 , piece_id=GNLESUJ3FCKKUC7HOR7Z532H7LQT6ZOTOKLBGRN2R6UCJL4S3VFQ, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.567576856 (251h1m54.298844507s) [noise=true , address=45.91.116.4:4952 , piece_id=ZXPM75JCQGD3XQ3BTIP4O6SOLR6XCFNBUYQKKD56RTFVCMEGFWDQ, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.575982240 (251h1m54.30724989s) [noise=true , address=77.90.31.19:4862 , piece_id=7CQOCX2FYKPQCTCBFMN7BMTD3FE2D2X4YIHFLCL22AOSYLKUO33A, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.576556950 (251h1m54.3078246s) [noise=true , address=77.90.31.17:4801 , piece_id=A4D6GQYALBOB46EE7D2KI6422NES4M43E4DTX4CKBLNNP2VHZUJQ, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.577052442 (251h1m54.308320102s) [noise=true , address=104.36.82.13:4204 , piece_id=LWFS4W2N7K6SWYKXYHZRJACAYYDBIB7OGVETZWKVTWWWJBZPS33A, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.577571397 (251h1m54.308839058s) [noise=true , address=104.36.82.12:4156 , piece_id=RMBTQYXVS5NKE2ON3WW5T2CBE6S7S6T74VVVLMUSM5C44UXGRAYA, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.581420654 (251h1m54.312688304s) [noise=true , address=77.90.31.12:4596 , piece_id=AEOFPZ64F5FUTWHLMKFVKQGNGYJYOIU7OZPLYGS7ULR52F5XSNYQ, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.582252959 (251h1m54.313520609s) [noise=true , address=77.90.31.18:4835 , piece_id=W6FEWYN4QCZC5IQFUG2YXSGKHSKKFRGETXK76LFKAVACLA37H7PA, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.672688665 (251h1m54.403956365s) [upload=261114]: Next returned nil segment Aug 19 04:28:13.672710426 (251h1m54.403978086s) [upload=261114]: Waiting for error group managing segments... Aug 19 04:28:13.681638350 (251h1m54.412906s) [noise=true , address=79.127.219.48:4085 , piece_id=AT2YETVN3DCMAPBL45XWQQCTW5FR6JC4PWN3CRIMZPQW5OG3RKKQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil> Aug 19 04:28:28.955286140 (251h2m9.68655377s) [noise=true , address=77.90.31.12:4596 , piece_id=AEOFPZ64F5FUTWHLMKFVKQGNGYJYOIU7OZPLYGS7ULR52F5XSNYQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1xgzDjjKfA3YT4tjGq6Esw9xfhA4xWMWnDbyAmNFUcP85qyFGf, address:77.90.31.12:4596): protocol: rpc: dial tcp 77.90.31.12:4596: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:45.285087265 (251h2m26.016354885s) [noise=true , address=104.36.82.60:4354 , piece_id=76HWEIKV5672IF34IRITARBMKGTN7OJU6GHQZZCDAP2ZUR2JNJFA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil> Aug 19 04:28:45.286367592 (251h2m26.017635222s) [noise=true , address=104.36.82.13:4204 , piece_id=LWFS4W2N7K6SWYKXYHZRJACAYYDBIB7OGVETZWKVTWWWJBZPS33A, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil> Aug 19 04:28:45.286429859 (251h2m26.017697489s) [noise=true , address=104.36.82.12:4156 , piece_id=RMBTQYXVS5NKE2ON3WW5T2CBE6S7S6T74VVVLMUSM5C44UXGRAYA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil> Aug 19 04:28:45.288231957 (251h2m26.019499578s) [noise=true , address=104.36.82.15:4289 , piece_id=GVBKZHWTPFDGYCM6U5EUYLBOTGCTTLPDWW2WZKZZEVNG4UZ3XEYA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil> Aug 19 04:28:45.316902561 (251h2m26.048170191s) [noise=true , address=104.36.82.44:4880 , piece_id=ZRDRV5AWRCYSUZJANVESK3XUCMLQQXF33WPV6QPX6HAZBYHXOUUQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil> Aug 19 04:28:46.178003461 (251h2m26.90927112s) [noise=true , address=104.36.82.5:4843 , piece_id=Q3SLO2FDTYYYYOQZARVS3SHAPM5J3GCLNFQ355QAV7U5MWKMZQQA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil> Aug 19 04:29:13.506175578 (251h2m54.237443199s) [noise=true , address=45.91.116.6:4988 , piece_id=X36K53BYKX53WMZKQWN2FFKCKTHNLIY3KLLPXMQN2UBKZM2CTT6Q, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1TrcoCHhAJeEykP9hJNpygoPSdgFV5v6GzL48JpyNPiAj5ZJRS, address:45.91.116.6:4988): protocol: rpc: dial tcp 45.91.116.6:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506222607 (251h2m54.237490227s) [noise=true , address=45.91.116.9:4286 , piece_id=KUMIYTCCEU7XS2NMUTPXDSD7TMNASOBWGNWNDD7L6DMEAPXVIFWA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:19DfbHig9tzF4VRHtSWkExwUS2bdZ8GFgKnAD9wvMBG7qaesNa, address:45.91.116.9:4286): protocol: rpc: dial tcp 45.91.116.9:4286: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506254087 (251h2m54.237521717s) [noise=true , address=45.91.116.12:4243 , piece_id=GNLESUJ3FCKKUC7HOR7Z532H7LQT6ZOTOKLBGRN2R6UCJL4S3VFQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1e9VEMmhYq6PxqvTpxsoUHzqYVwujfj1UG7HSZFeiu8CderMpM, address:45.91.116.12:4243): protocol: rpc: dial tcp 45.91.116.12:4243: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506625304 (251h2m54.237892934s) [noise=true , address=77.90.31.21:4891 , piece_id=G57I4V2AOHDBI77GIIULZXTGCFC2BP6CQRX25KA7UNNIJ3WTAGPQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:17RDAerfx8AW5NsgWLTXLNDbuAEWQFT3SVM4iwdmuBzWrML83g, address:77.90.31.21:4891): protocol: rpc: dial tcp 77.90.31.21:4891: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506656563 (251h2m54.237924213s) [noise=true , address=77.90.31.19:4862 , piece_id=7CQOCX2FYKPQCTCBFMN7BMTD3FE2D2X4YIHFLCL22AOSYLKUO33A, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1Tvmo3nSgBPcFYW9MmZQGUeMoD3m3ykJHwVMj6tj69iQKgX3Mz, address:77.90.31.19:4862): protocol: rpc: dial tcp 77.90.31.19:4862: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 19 04:29:13.506689415 (251h2m54.237957045s) [noise=true , address=77.90.31.18:4835 , piece_id=W6FEWYN4QCZC5IQFUG2YXSGKHSKKFRGETXK76LFKAVACLA37H7PA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:12hAQp2Zr7WSaQSTCZf5pwSyuBHHSrZWq5EVz5hFvSA5ddWebqg, address:77.90.31.18:4835): protocol: write tcp 5.161.89.252:27410->77.90.31.18:4835: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506740311 (251h2m54.23800793s) [noise=true , address=45.91.116.7:4037 , piece_id=B42RTCUIHRS5OUFJ5S5E6QWRKEN5H34IMBURUUSSUL53Y2FUYKZQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1iNPwE2XSfXcaXAXcrH2jcyNGYeDiiWvzAcDQffrmjPwTVqVmK, address:45.91.116.7:4037): protocol: rpc: dial tcp 45.91.116.7:4037: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506774885 (251h2m54.238042535s) [noise=true , address=77.90.31.17:4801 , piece_id=A4D6GQYALBOB46EE7D2KI6422NES4M43E4DTX4CKBLNNP2VHZUJQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:127QRLNwZLcR5gnAgZrCjSUsH5T4jESmZ4kWeAnpj9YxaTSjoYe, address:77.90.31.17:4801): protocol: write tcp 5.161.89.252:48394->77.90.31.17:4801: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506808788 (251h2m54.238076419s) [noise=true , address=45.91.116.4:4952 , piece_id=ZXPM75JCQGD3XQ3BTIP4O6SOLR6XCFNBUYQKKD56RTFVCMEGFWDQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1PcGuaN6uLz1HVzMgBp4vViKJw2KrmaFatL1w9DHgiYqccMUJ9, address:45.91.116.4:4952): protocol: write tcp 5.161.89.252:61300->45.91.116.4:4952: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506825761 (251h2m54.23809341s) [noise=true , address=77.90.31.13:4645 , piece_id=TGFZGBBRZLA32ZJS3BGT7OSHJW4S34PWMYSQKCZT4RG2IUTGPZSQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg, address:77.90.31.13:4645): protocol: rpc: dial tcp 77.90.31.13:4645: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506906624 (251h2m54.238174313s) [seg_pos={0 0}, upload=261114]: Done waiting for segment. successful: 7 optimal: 14 errs: <nil> Aug 19 04:29:13.523807608 (251h2m54.255075238s) [upload=261114]: Done uploading segments... Aug 19 04:29:13.523824049 (251h2m54.255091679s) [upload=261114]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 7) Duration: 1.312950117s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 9.660878ms target: http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt FAIL (2025-08-19 05:00:08 UTC) orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp: lookup link.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:34965->127.0.0.1:53: i/o timeout Duration: 9.679363ms edge linksharing us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 246.152176ms FAIL (2025-08-19 05:00:07 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:26171->127.0.0.1:53: i/o timeout Duration: 246.171682ms edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 255.406399ms FAIL (2025-08-19 05:00:07 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:26171->127.0.0.1:53: i/o timeout Duration: 255.426837ms edge authservice http us-select-1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 703.060924ms FAIL (2025-08-19 04:59:56 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:34390->127.0.0.1:53: i/o timeout Duration: 703.087003ms edge authservice http bgp ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 4.738867518s FAIL (2025-08-19 04:59:54 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 4.738911602s core uplink ap1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 218.456459ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 FAIL (2025-08-19 04:59:52 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:41477->127.0.0.1:53: i/o timeout Duration: 218.482117ms edge gateway vhost bgp eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 874.992536ms FAIL (2025-08-19 04:59:51 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 875.022832ms core uplink eu1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 40.996806ms FAIL (2025-08-19 04:59:49 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:26171->127.0.0.1:53: i/o timeout Duration: 41.024447ms edge authservice http us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 385.506768ms FAIL (2025-08-19 04:59:44 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:26171->127.0.0.1:53: i/o timeout Duration: 385.542366ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [............XXXXXX..........................................] duration: 1.011420965s FAIL (2025-08-19 04:59:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ad=261108]: Uploading piece... Aug 19 04:27:59.298715587 (251h1m40.029983236s) [noise=true , address=45.91.116.10:4081 , piece_id=VXW7T7RYDQYFA67EV4IP5EFMNLSPHO6YMYO4NCUBCDOSPQWID6KQ, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.298767684 (251h1m40.030035364s) [noise=true , address=45.91.116.11:4119 , piece_id=6NZDOUCLI5JRQ5XXZMNLBXIVXMHR7NJY6BTCMA22QMNC3ZJJFCMA, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.299257736 (251h1m40.030525387s) [noise=true , address=45.91.116.10:4083 , piece_id=N7ABFK4IGEFBUNEAQI3SFS7IC6SCOHQD4TN2QOWDUSNRM4ZLXV7A, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.299339650 (251h1m40.030607301s) [noise=true , address=77.90.31.21:4878 , piece_id=X3FKRFUU6S3AB6ZPUNPB2HOGLJ5E7U4AWTEFH6PISRIBECP3GVSQ, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.299714204 (251h1m40.030981855s) [noise=true , address=45.91.116.8:4045 , piece_id=OH3KNS4URAIN3T4FLAPSYC2SHQKOEBLJMF2AVG7MXJULLC3P7QYA, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.299820754 (251h1m40.031088415s) [noise=true , address=77.90.31.10:4517 , piece_id=QXH4JBJTSA4VLHPS2OUAR3XF7QEL3WSX2JJT7VTZMRBVWS3OIOQQ, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.300136579 (251h1m40.031404229s) [noise=true , address=77.90.31.12:4592 , piece_id=25GKL2MHGGV6IC7JDU5MBJ4MOCCNPGMXQPYXH7A4FFFJ5EZGVZ2Q, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.300491886 (251h1m40.031759547s) [noise=true , address=45.91.116.9:4261 , piece_id=RAEAWIKMHIV4VGOZY57ZK36BNZGZ2ATXTXC4YHRABT6VWB6MGILA, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.300854618 (251h1m40.032122269s) [noise=true , address=104.36.82.2:4732 , piece_id=WKRORDL2XPRF2KRKZ55EQ2PTPJHPJSQ6NQ4D3VVFK6COGG57KDNQ, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.301277543 (251h1m40.032545194s) [noise=true , address=45.91.116.14:4190 , piece_id=SGZNGKGYEKPU4GXCULPKYIOONZZXZT6UTO7MDWYOJRJJW4WSG3VA, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.301655164 (251h1m40.032922814s) [noise=true , address=104.36.82.7:4910 , piece_id=R6GR56GOKT2K3AX75OL3BWODTZ4FJY27MIOQWFYJVY4UGR66DIBQ, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.302072699 (251h1m40.033340349s) [noise=true , address=104.36.82.16:4294 , piece_id=GT3KQOCWD2YTWBQDAO5QJKYIMZINUF4MESKP6F3P6J73R4IY3NDQ, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:28:02.354368898 (251h1m43.085636557s) [noise=true , address=77.90.31.21:4878 , piece_id=X3FKRFUU6S3AB6ZPUNPB2HOGLJ5E7U4AWTEFH6PISRIBECP3GVSQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: write tcp 5.161.89.252:64042->77.90.31.21:4878: write: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:02.385567243 (251h1m43.116834893s) [noise=true , address=77.90.31.12:4592 , piece_id=25GKL2MHGGV6IC7JDU5MBJ4MOCCNPGMXQPYXH7A4FFFJ5EZGVZ2Q, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 5.161.89.252:34306->77.90.31.12:4592: write: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Aug 19 04:28:29.225676153 (251h2m9.956943773s) [upload=261108]: Next segment err: context deadline exceeded Aug 19 04:28:29.225998790 (251h2m9.95726642s) [noise=true , address=104.36.82.2:4732 , piece_id=WKRORDL2XPRF2KRKZ55EQ2PTPJHPJSQ6NQ4D3VVFK6COGG57KDNQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226039707 (251h2m9.957307326s) [noise=true , address=45.91.116.11:4119 , piece_id=6NZDOUCLI5JRQ5XXZMNLBXIVXMHR7NJY6BTCMA22QMNC3ZJJFCMA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226072177 (251h2m9.957339798s) [noise=true , address=45.91.116.8:4045 , piece_id=OH3KNS4URAIN3T4FLAPSYC2SHQKOEBLJMF2AVG7MXJULLC3P7QYA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226223943 (251h2m9.957491573s) [noise=true , address=45.91.116.14:4190 , piece_id=SGZNGKGYEKPU4GXCULPKYIOONZZXZT6UTO7MDWYOJRJJW4WSG3VA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226274027 (251h2m9.957541647s) [noise=true , address=45.91.116.13:4154 , piece_id=WM56E23Z7FLRF33HYDXONXHODJDHPCJFWGVC3NI7RUTZUESWO4MQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226310786 (251h2m9.957578416s) [noise=true , address=77.90.31.17:4771 , piece_id=3BVZK3ZL7N5FC3OJGFFYTVEG7INVMLMG2QBPXX3UC3EJVXHBGW3Q, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226348096 (251h2m9.957615726s) [noise=true , address=104.36.82.11:4114 , piece_id=P4RJRUZPRTFPRK5WWHKUVEKJKXGPPPCIV4RPFIZOBO2CPOXSTSIQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: rpc: dial tcp 104.36.82.11:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226392119 (251h2m9.957659749s) [noise=true , address=45.91.116.10:4081 , piece_id=VXW7T7RYDQYFA67EV4IP5EFMNLSPHO6YMYO4NCUBCDOSPQWID6KQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 5.161.89.252:44604->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226421564 (251h2m9.957689195s) [noise=true , address=45.91.116.9:4261 , piece_id=RAEAWIKMHIV4VGOZY57ZK36BNZGZ2ATXTXC4YHRABT6VWB6MGILA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226443746 (251h2m9.957711377s) [noise=true , address=45.91.116.4:4940 , piece_id=GYF7CH3463KOWU7DE7WKLYTXYXLTRMPOIK4K3D64ANKOERN33FGQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 5.161.89.252:53922->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226474684 (251h2m9.957742315s) [noise=true , address=104.36.82.7:4910 , piece_id=R6GR56GOKT2K3AX75OL3BWODTZ4FJY27MIOQWFYJVY4UGR66DIBQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226493810 (251h2m9.95776143s) [noise=true , address=45.91.116.10:4083 , piece_id=N7ABFK4IGEFBUNEAQI3SFS7IC6SCOHQD4TN2QOWDUSNRM4ZLXV7A, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: rpc: dial tcp 45.91.116.10:4083: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226509199 (251h2m9.957776829s) [noise=true , address=104.36.82.16:4294 , piece_id=GT3KQOCWD2YTWBQDAO5QJKYIMZINUF4MESKP6F3P6J73R4IY3NDQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:104.36.82.16:4294): protocol: write tcp 5.161.89.252:25886->104.36.82.16:4294: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226527894 (251h2m9.957795524s) [noise=true , address=77.90.31.10:4517 , piece_id=QXH4JBJTSA4VLHPS2OUAR3XF7QEL3WSX2JJT7VTZMRBVWS3OIOQQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226544845 (251h2m9.957812476s) [seg_pos={0 0}, upload=261108]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 1.011441374s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 112.552437ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-19 04:59:43 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:41477->127.0.0.1:53: i/o timeout Duration: 112.56439ms edge gateway vhost bgp us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 1.833690623s FAIL (2025-08-19 04:59:42 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:34390->127.0.0.1:53: i/o timeout Duration: 1.833710299s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............XXXX.........................................] duration: 606.368503ms FAIL (2025-08-19 04:59:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 7) ======LOGS====== g piece... Aug 19 04:28:13.563886368 (251h1m54.295154018s) [noise=true , address=104.36.82.5:4843 , piece_id=Q3SLO2FDTYYYYOQZARVS3SHAPM5J3GCLNFQ355QAV7U5MWKMZQQA, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.564531581 (251h1m54.295799231s) [noise=true , address=45.91.116.7:4037 , piece_id=B42RTCUIHRS5OUFJ5S5E6QWRKEN5H34IMBURUUSSUL53Y2FUYKZQ, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.564716169 (251h1m54.295983828s) [noise=true , address=104.36.82.15:4289 , piece_id=GVBKZHWTPFDGYCM6U5EUYLBOTGCTTLPDWW2WZKZZEVNG4UZ3XEYA, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.565358756 (251h1m54.296626407s) [noise=true , address=77.90.31.21:4891 , piece_id=G57I4V2AOHDBI77GIIULZXTGCFC2BP6CQRX25KA7UNNIJ3WTAGPQ, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.565475757 (251h1m54.296743417s) [noise=true , address=45.91.116.9:4286 , piece_id=KUMIYTCCEU7XS2NMUTPXDSD7TMNASOBWGNWNDD7L6DMEAPXVIFWA, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.566262315 (251h1m54.297529975s) [noise=true , address=104.36.82.44:4880 , piece_id=ZRDRV5AWRCYSUZJANVESK3XUCMLQQXF33WPV6QPX6HAZBYHXOUUQ, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.566506594 (251h1m54.297774285s) [noise=true , address=45.91.116.12:4243 , piece_id=GNLESUJ3FCKKUC7HOR7Z532H7LQT6ZOTOKLBGRN2R6UCJL4S3VFQ, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.567576856 (251h1m54.298844507s) [noise=true , address=45.91.116.4:4952 , piece_id=ZXPM75JCQGD3XQ3BTIP4O6SOLR6XCFNBUYQKKD56RTFVCMEGFWDQ, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.575982240 (251h1m54.30724989s) [noise=true , address=77.90.31.19:4862 , piece_id=7CQOCX2FYKPQCTCBFMN7BMTD3FE2D2X4YIHFLCL22AOSYLKUO33A, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.576556950 (251h1m54.3078246s) [noise=true , address=77.90.31.17:4801 , piece_id=A4D6GQYALBOB46EE7D2KI6422NES4M43E4DTX4CKBLNNP2VHZUJQ, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.577052442 (251h1m54.308320102s) [noise=true , address=104.36.82.13:4204 , piece_id=LWFS4W2N7K6SWYKXYHZRJACAYYDBIB7OGVETZWKVTWWWJBZPS33A, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.577571397 (251h1m54.308839058s) [noise=true , address=104.36.82.12:4156 , piece_id=RMBTQYXVS5NKE2ON3WW5T2CBE6S7S6T74VVVLMUSM5C44UXGRAYA, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.581420654 (251h1m54.312688304s) [noise=true , address=77.90.31.12:4596 , piece_id=AEOFPZ64F5FUTWHLMKFVKQGNGYJYOIU7OZPLYGS7ULR52F5XSNYQ, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.582252959 (251h1m54.313520609s) [noise=true , address=77.90.31.18:4835 , piece_id=W6FEWYN4QCZC5IQFUG2YXSGKHSKKFRGETXK76LFKAVACLA37H7PA, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.672688665 (251h1m54.403956365s) [upload=261114]: Next returned nil segment Aug 19 04:28:13.672710426 (251h1m54.403978086s) [upload=261114]: Waiting for error group managing segments... Aug 19 04:28:13.681638350 (251h1m54.412906s) [noise=true , address=79.127.219.48:4085 , piece_id=AT2YETVN3DCMAPBL45XWQQCTW5FR6JC4PWN3CRIMZPQW5OG3RKKQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil> Aug 19 04:28:28.955286140 (251h2m9.68655377s) [noise=true , address=77.90.31.12:4596 , piece_id=AEOFPZ64F5FUTWHLMKFVKQGNGYJYOIU7OZPLYGS7ULR52F5XSNYQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1xgzDjjKfA3YT4tjGq6Esw9xfhA4xWMWnDbyAmNFUcP85qyFGf, address:77.90.31.12:4596): protocol: rpc: dial tcp 77.90.31.12:4596: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:45.285087265 (251h2m26.016354885s) [noise=true , address=104.36.82.60:4354 , piece_id=76HWEIKV5672IF34IRITARBMKGTN7OJU6GHQZZCDAP2ZUR2JNJFA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil> Aug 19 04:28:45.286367592 (251h2m26.017635222s) [noise=true , address=104.36.82.13:4204 , piece_id=LWFS4W2N7K6SWYKXYHZRJACAYYDBIB7OGVETZWKVTWWWJBZPS33A, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil> Aug 19 04:28:45.286429859 (251h2m26.017697489s) [noise=true , address=104.36.82.12:4156 , piece_id=RMBTQYXVS5NKE2ON3WW5T2CBE6S7S6T74VVVLMUSM5C44UXGRAYA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil> Aug 19 04:28:45.288231957 (251h2m26.019499578s) [noise=true , address=104.36.82.15:4289 , piece_id=GVBKZHWTPFDGYCM6U5EUYLBOTGCTTLPDWW2WZKZZEVNG4UZ3XEYA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil> Aug 19 04:28:45.316902561 (251h2m26.048170191s) [noise=true , address=104.36.82.44:4880 , piece_id=ZRDRV5AWRCYSUZJANVESK3XUCMLQQXF33WPV6QPX6HAZBYHXOUUQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil> Aug 19 04:28:46.178003461 (251h2m26.90927112s) [noise=true , address=104.36.82.5:4843 , piece_id=Q3SLO2FDTYYYYOQZARVS3SHAPM5J3GCLNFQ355QAV7U5MWKMZQQA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil> Aug 19 04:29:13.506175578 (251h2m54.237443199s) [noise=true , address=45.91.116.6:4988 , piece_id=X36K53BYKX53WMZKQWN2FFKCKTHNLIY3KLLPXMQN2UBKZM2CTT6Q, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1TrcoCHhAJeEykP9hJNpygoPSdgFV5v6GzL48JpyNPiAj5ZJRS, address:45.91.116.6:4988): protocol: rpc: dial tcp 45.91.116.6:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506222607 (251h2m54.237490227s) [noise=true , address=45.91.116.9:4286 , piece_id=KUMIYTCCEU7XS2NMUTPXDSD7TMNASOBWGNWNDD7L6DMEAPXVIFWA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:19DfbHig9tzF4VRHtSWkExwUS2bdZ8GFgKnAD9wvMBG7qaesNa, address:45.91.116.9:4286): protocol: rpc: dial tcp 45.91.116.9:4286: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506254087 (251h2m54.237521717s) [noise=true , address=45.91.116.12:4243 , piece_id=GNLESUJ3FCKKUC7HOR7Z532H7LQT6ZOTOKLBGRN2R6UCJL4S3VFQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1e9VEMmhYq6PxqvTpxsoUHzqYVwujfj1UG7HSZFeiu8CderMpM, address:45.91.116.12:4243): protocol: rpc: dial tcp 45.91.116.12:4243: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506625304 (251h2m54.237892934s) [noise=true , address=77.90.31.21:4891 , piece_id=G57I4V2AOHDBI77GIIULZXTGCFC2BP6CQRX25KA7UNNIJ3WTAGPQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:17RDAerfx8AW5NsgWLTXLNDbuAEWQFT3SVM4iwdmuBzWrML83g, address:77.90.31.21:4891): protocol: rpc: dial tcp 77.90.31.21:4891: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506656563 (251h2m54.237924213s) [noise=true , address=77.90.31.19:4862 , piece_id=7CQOCX2FYKPQCTCBFMN7BMTD3FE2D2X4YIHFLCL22AOSYLKUO33A, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1Tvmo3nSgBPcFYW9MmZQGUeMoD3m3ykJHwVMj6tj69iQKgX3Mz, address:77.90.31.19:4862): protocol: rpc: dial tcp 77.90.31.19:4862: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 19 04:29:13.506689415 (251h2m54.237957045s) [noise=true , address=77.90.31.18:4835 , piece_id=W6FEWYN4QCZC5IQFUG2YXSGKHSKKFRGETXK76LFKAVACLA37H7PA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:12hAQp2Zr7WSaQSTCZf5pwSyuBHHSrZWq5EVz5hFvSA5ddWebqg, address:77.90.31.18:4835): protocol: write tcp 5.161.89.252:27410->77.90.31.18:4835: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506740311 (251h2m54.23800793s) [noise=true , address=45.91.116.7:4037 , piece_id=B42RTCUIHRS5OUFJ5S5E6QWRKEN5H34IMBURUUSSUL53Y2FUYKZQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1iNPwE2XSfXcaXAXcrH2jcyNGYeDiiWvzAcDQffrmjPwTVqVmK, address:45.91.116.7:4037): protocol: rpc: dial tcp 45.91.116.7:4037: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506774885 (251h2m54.238042535s) [noise=true , address=77.90.31.17:4801 , piece_id=A4D6GQYALBOB46EE7D2KI6422NES4M43E4DTX4CKBLNNP2VHZUJQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:127QRLNwZLcR5gnAgZrCjSUsH5T4jESmZ4kWeAnpj9YxaTSjoYe, address:77.90.31.17:4801): protocol: write tcp 5.161.89.252:48394->77.90.31.17:4801: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506808788 (251h2m54.238076419s) [noise=true , address=45.91.116.4:4952 , piece_id=ZXPM75JCQGD3XQ3BTIP4O6SOLR6XCFNBUYQKKD56RTFVCMEGFWDQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1PcGuaN6uLz1HVzMgBp4vViKJw2KrmaFatL1w9DHgiYqccMUJ9, address:45.91.116.4:4952): protocol: write tcp 5.161.89.252:61300->45.91.116.4:4952: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506825761 (251h2m54.23809341s) [noise=true , address=77.90.31.13:4645 , piece_id=TGFZGBBRZLA32ZJS3BGT7OSHJW4S34PWMYSQKCZT4RG2IUTGPZSQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg, address:77.90.31.13:4645): protocol: rpc: dial tcp 77.90.31.13:4645: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506906624 (251h2m54.238174313s) [seg_pos={0 0}, upload=261114]: Done waiting for segment. successful: 7 optimal: 14 errs: <nil> Aug 19 04:29:13.523807608 (251h2m54.255075238s) [upload=261114]: Done uploading segments... Aug 19 04:29:13.523824049 (251h2m54.255091679s) [upload=261114]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 7) Duration: 606.390284ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [................XXXX.................X......................] duration: 544.276333ms FAIL (2025-08-19 04:59:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:42:23.281867426 (251h16m4.013135076s) [upload=261352]: Starting upload Aug 19 04:42:23.281936796 (251h16m4.013204446s) [upload=261352]: Done starting upload Aug 19 04:42:23.282094062 (251h16m4.013361712s) [upload=261352]: Uploading segments... Aug 19 04:42:23.282376082 (251h16m4.013643742s) [upload=261352]: Got next segment. Inline: false Aug 19 04:42:23.282443619 (251h16m4.013711269s) [upload=261352]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 04:42:23.282451122 (251h16m4.013718773s) [upload=261352]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 04:42:23.284629308 (251h16m4.015896958s) [upload=261352]: Next segment err: metaclient: Too Many Requests Aug 19 04:42:23.284645568 (251h16m4.015913219s) [upload=261352]: Done uploading segments... Aug 19 04:42:23.284659365 (251h16m4.015927014s) [upload=261352]: Upload finished. err: metaclient: Too Many Requests Duration: 544.296642ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 925.600618ms FAIL (2025-08-19 04:59:33 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 925.619604ms core uplink eu1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 209.147531ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 FAIL (2025-08-19 04:59:20 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:41477->127.0.0.1:53: i/o timeout Duration: 209.17333ms edge gateway vhost bgp eu1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 895.978703ms FAIL (2025-08-19 04:59:20 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:34390->127.0.0.1:53: i/o timeout Duration: 896.006956ms edge authservice http bgp ap1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 1.381681453s FAIL (2025-08-19 04:59:13 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:34390->127.0.0.1:53: i/o timeout Duration: 1.381699897s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 695.619889ms target: https://ap1.storj.io/login FAIL (2025-08-19 04:59:09 UTC) orbiter warning: Get "https://ap1.storj.io/login": dial tcp 35.213.185.92:443: i/o timeout Duration: 695.637533ms core satellite ap1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 607.686544ms FAIL (2025-08-19 04:59:09 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:34390->127.0.0.1:53: i/o timeout Duration: 607.706833ms edge authservice http bgp eu1
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 13.629418ms target: http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt FAIL (2025-08-19 04:59:07 UTC) orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp: lookup link.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:34965->127.0.0.1:53: i/o timeout Duration: 13.650528ms edge linksharing us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 167.157538ms FAIL (2025-08-19 04:58:58 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:26171->127.0.0.1:53: i/o timeout Duration: 167.181113ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.................XXXX.................X.....................] duration: 520.736859ms FAIL (2025-08-19 04:58:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 04:42:23.281867426 (251h16m4.013135076s) [upload=261352]: Starting upload Aug 19 04:42:23.281936796 (251h16m4.013204446s) [upload=261352]: Done starting upload Aug 19 04:42:23.282094062 (251h16m4.013361712s) [upload=261352]: Uploading segments... Aug 19 04:42:23.282376082 (251h16m4.013643742s) [upload=261352]: Got next segment. Inline: false Aug 19 04:42:23.282443619 (251h16m4.013711269s) [upload=261352]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 04:42:23.282451122 (251h16m4.013718773s) [upload=261352]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 04:42:23.284629308 (251h16m4.015896958s) [upload=261352]: Next segment err: metaclient: Too Many Requests Aug 19 04:42:23.284645568 (251h16m4.015913219s) [upload=261352]: Done uploading segments... Aug 19 04:42:23.284659365 (251h16m4.015927014s) [upload=261352]: Upload finished. err: metaclient: Too Many Requests Duration: 520.764081ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 358.547255ms FAIL (2025-08-19 04:58:56 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:26171->127.0.0.1:53: i/o timeout Duration: 358.568264ms edge gateway us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 4.81938436s FAIL (2025-08-19 04:58:54 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 4.81941151s core uplink ap1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 912.75345ms FAIL (2025-08-19 04:58:47 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 912.789367ms core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.............XXXXXX.........................................] duration: 525.216768ms FAIL (2025-08-19 04:58:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ad=261108]: Uploading piece... Aug 19 04:27:59.298715587 (251h1m40.029983236s) [noise=true , address=45.91.116.10:4081 , piece_id=VXW7T7RYDQYFA67EV4IP5EFMNLSPHO6YMYO4NCUBCDOSPQWID6KQ, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.298767684 (251h1m40.030035364s) [noise=true , address=45.91.116.11:4119 , piece_id=6NZDOUCLI5JRQ5XXZMNLBXIVXMHR7NJY6BTCMA22QMNC3ZJJFCMA, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.299257736 (251h1m40.030525387s) [noise=true , address=45.91.116.10:4083 , piece_id=N7ABFK4IGEFBUNEAQI3SFS7IC6SCOHQD4TN2QOWDUSNRM4ZLXV7A, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.299339650 (251h1m40.030607301s) [noise=true , address=77.90.31.21:4878 , piece_id=X3FKRFUU6S3AB6ZPUNPB2HOGLJ5E7U4AWTEFH6PISRIBECP3GVSQ, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.299714204 (251h1m40.030981855s) [noise=true , address=45.91.116.8:4045 , piece_id=OH3KNS4URAIN3T4FLAPSYC2SHQKOEBLJMF2AVG7MXJULLC3P7QYA, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.299820754 (251h1m40.031088415s) [noise=true , address=77.90.31.10:4517 , piece_id=QXH4JBJTSA4VLHPS2OUAR3XF7QEL3WSX2JJT7VTZMRBVWS3OIOQQ, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.300136579 (251h1m40.031404229s) [noise=true , address=77.90.31.12:4592 , piece_id=25GKL2MHGGV6IC7JDU5MBJ4MOCCNPGMXQPYXH7A4FFFJ5EZGVZ2Q, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.300491886 (251h1m40.031759547s) [noise=true , address=45.91.116.9:4261 , piece_id=RAEAWIKMHIV4VGOZY57ZK36BNZGZ2ATXTXC4YHRABT6VWB6MGILA, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.300854618 (251h1m40.032122269s) [noise=true , address=104.36.82.2:4732 , piece_id=WKRORDL2XPRF2KRKZ55EQ2PTPJHPJSQ6NQ4D3VVFK6COGG57KDNQ, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.301277543 (251h1m40.032545194s) [noise=true , address=45.91.116.14:4190 , piece_id=SGZNGKGYEKPU4GXCULPKYIOONZZXZT6UTO7MDWYOJRJJW4WSG3VA, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.301655164 (251h1m40.032922814s) [noise=true , address=104.36.82.7:4910 , piece_id=R6GR56GOKT2K3AX75OL3BWODTZ4FJY27MIOQWFYJVY4UGR66DIBQ, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:27:59.302072699 (251h1m40.033340349s) [noise=true , address=104.36.82.16:4294 , piece_id=GT3KQOCWD2YTWBQDAO5QJKYIMZINUF4MESKP6F3P6J73R4IY3NDQ, seg_pos={0 0}, upload=261108]: Uploading piece... Aug 19 04:28:02.354368898 (251h1m43.085636557s) [noise=true , address=77.90.31.21:4878 , piece_id=X3FKRFUU6S3AB6ZPUNPB2HOGLJ5E7U4AWTEFH6PISRIBECP3GVSQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: write tcp 5.161.89.252:64042->77.90.31.21:4878: write: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:02.385567243 (251h1m43.116834893s) [noise=true , address=77.90.31.12:4592 , piece_id=25GKL2MHGGV6IC7JDU5MBJ4MOCCNPGMXQPYXH7A4FFFJ5EZGVZ2Q, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 5.161.89.252:34306->77.90.31.12:4592: write: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Aug 19 04:28:29.225676153 (251h2m9.956943773s) [upload=261108]: Next segment err: context deadline exceeded Aug 19 04:28:29.225998790 (251h2m9.95726642s) [noise=true , address=104.36.82.2:4732 , piece_id=WKRORDL2XPRF2KRKZ55EQ2PTPJHPJSQ6NQ4D3VVFK6COGG57KDNQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226039707 (251h2m9.957307326s) [noise=true , address=45.91.116.11:4119 , piece_id=6NZDOUCLI5JRQ5XXZMNLBXIVXMHR7NJY6BTCMA22QMNC3ZJJFCMA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226072177 (251h2m9.957339798s) [noise=true , address=45.91.116.8:4045 , piece_id=OH3KNS4URAIN3T4FLAPSYC2SHQKOEBLJMF2AVG7MXJULLC3P7QYA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226223943 (251h2m9.957491573s) [noise=true , address=45.91.116.14:4190 , piece_id=SGZNGKGYEKPU4GXCULPKYIOONZZXZT6UTO7MDWYOJRJJW4WSG3VA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226274027 (251h2m9.957541647s) [noise=true , address=45.91.116.13:4154 , piece_id=WM56E23Z7FLRF33HYDXONXHODJDHPCJFWGVC3NI7RUTZUESWO4MQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226310786 (251h2m9.957578416s) [noise=true , address=77.90.31.17:4771 , piece_id=3BVZK3ZL7N5FC3OJGFFYTVEG7INVMLMG2QBPXX3UC3EJVXHBGW3Q, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226348096 (251h2m9.957615726s) [noise=true , address=104.36.82.11:4114 , piece_id=P4RJRUZPRTFPRK5WWHKUVEKJKXGPPPCIV4RPFIZOBO2CPOXSTSIQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: rpc: dial tcp 104.36.82.11:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226392119 (251h2m9.957659749s) [noise=true , address=45.91.116.10:4081 , piece_id=VXW7T7RYDQYFA67EV4IP5EFMNLSPHO6YMYO4NCUBCDOSPQWID6KQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 5.161.89.252:44604->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226421564 (251h2m9.957689195s) [noise=true , address=45.91.116.9:4261 , piece_id=RAEAWIKMHIV4VGOZY57ZK36BNZGZ2ATXTXC4YHRABT6VWB6MGILA, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226443746 (251h2m9.957711377s) [noise=true , address=45.91.116.4:4940 , piece_id=GYF7CH3463KOWU7DE7WKLYTXYXLTRMPOIK4K3D64ANKOERN33FGQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 5.161.89.252:53922->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226474684 (251h2m9.957742315s) [noise=true , address=104.36.82.7:4910 , piece_id=R6GR56GOKT2K3AX75OL3BWODTZ4FJY27MIOQWFYJVY4UGR66DIBQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226493810 (251h2m9.95776143s) [noise=true , address=45.91.116.10:4083 , piece_id=N7ABFK4IGEFBUNEAQI3SFS7IC6SCOHQD4TN2QOWDUSNRM4ZLXV7A, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: rpc: dial tcp 45.91.116.10:4083: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226509199 (251h2m9.957776829s) [noise=true , address=104.36.82.16:4294 , piece_id=GT3KQOCWD2YTWBQDAO5QJKYIMZINUF4MESKP6F3P6J73R4IY3NDQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:104.36.82.16:4294): protocol: write tcp 5.161.89.252:25886->104.36.82.16:4294: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226527894 (251h2m9.957795524s) [noise=true , address=77.90.31.10:4517 , piece_id=QXH4JBJTSA4VLHPS2OUAR3XF7QEL3WSX2JJT7VTZMRBVWS3OIOQQ, seg_pos={0 0}, upload=261108]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:29.226544845 (251h2m9.957812476s) [seg_pos={0 0}, upload=261108]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 525.242296ms core uplink us-select-1
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 678.761463ms target: https://ap1.storj.io/login FAIL (2025-08-19 04:58:40 UTC) orbiter warning: Get "https://ap1.storj.io/login": dial tcp 35.213.185.92:443: i/o timeout Duration: 678.780619ms core satellite ap1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 42.974844ms FAIL (2025-08-19 04:58:37 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:26171->127.0.0.1:53: i/o timeout Duration: 42.996896ms edge authservice http us-select-1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 695.620987ms FAIL (2025-08-19 04:58:37 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:34390->127.0.0.1:53: i/o timeout Duration: 695.646887ms edge authservice http bgp ap1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 111.121848ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-19 04:58:34 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:41477->127.0.0.1:53: i/o timeout Duration: 111.155282ms edge gateway vhost bgp us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 1.320625809s FAIL (2025-08-19 04:58:34 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:34390->127.0.0.1:53: i/o timeout Duration: 1.32064739s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 470.873448ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 FAIL (2025-08-19 04:58:33 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:41477->127.0.0.1:53: i/o timeout Duration: 470.893485ms edge gateway vhost bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [................XXXX........................................] duration: 530.042679ms FAIL (2025-08-19 04:58:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 7) ======LOGS====== g piece... Aug 19 04:28:13.563886368 (251h1m54.295154018s) [noise=true , address=104.36.82.5:4843 , piece_id=Q3SLO2FDTYYYYOQZARVS3SHAPM5J3GCLNFQ355QAV7U5MWKMZQQA, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.564531581 (251h1m54.295799231s) [noise=true , address=45.91.116.7:4037 , piece_id=B42RTCUIHRS5OUFJ5S5E6QWRKEN5H34IMBURUUSSUL53Y2FUYKZQ, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.564716169 (251h1m54.295983828s) [noise=true , address=104.36.82.15:4289 , piece_id=GVBKZHWTPFDGYCM6U5EUYLBOTGCTTLPDWW2WZKZZEVNG4UZ3XEYA, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.565358756 (251h1m54.296626407s) [noise=true , address=77.90.31.21:4891 , piece_id=G57I4V2AOHDBI77GIIULZXTGCFC2BP6CQRX25KA7UNNIJ3WTAGPQ, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.565475757 (251h1m54.296743417s) [noise=true , address=45.91.116.9:4286 , piece_id=KUMIYTCCEU7XS2NMUTPXDSD7TMNASOBWGNWNDD7L6DMEAPXVIFWA, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.566262315 (251h1m54.297529975s) [noise=true , address=104.36.82.44:4880 , piece_id=ZRDRV5AWRCYSUZJANVESK3XUCMLQQXF33WPV6QPX6HAZBYHXOUUQ, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.566506594 (251h1m54.297774285s) [noise=true , address=45.91.116.12:4243 , piece_id=GNLESUJ3FCKKUC7HOR7Z532H7LQT6ZOTOKLBGRN2R6UCJL4S3VFQ, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.567576856 (251h1m54.298844507s) [noise=true , address=45.91.116.4:4952 , piece_id=ZXPM75JCQGD3XQ3BTIP4O6SOLR6XCFNBUYQKKD56RTFVCMEGFWDQ, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.575982240 (251h1m54.30724989s) [noise=true , address=77.90.31.19:4862 , piece_id=7CQOCX2FYKPQCTCBFMN7BMTD3FE2D2X4YIHFLCL22AOSYLKUO33A, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.576556950 (251h1m54.3078246s) [noise=true , address=77.90.31.17:4801 , piece_id=A4D6GQYALBOB46EE7D2KI6422NES4M43E4DTX4CKBLNNP2VHZUJQ, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.577052442 (251h1m54.308320102s) [noise=true , address=104.36.82.13:4204 , piece_id=LWFS4W2N7K6SWYKXYHZRJACAYYDBIB7OGVETZWKVTWWWJBZPS33A, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.577571397 (251h1m54.308839058s) [noise=true , address=104.36.82.12:4156 , piece_id=RMBTQYXVS5NKE2ON3WW5T2CBE6S7S6T74VVVLMUSM5C44UXGRAYA, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.581420654 (251h1m54.312688304s) [noise=true , address=77.90.31.12:4596 , piece_id=AEOFPZ64F5FUTWHLMKFVKQGNGYJYOIU7OZPLYGS7ULR52F5XSNYQ, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.582252959 (251h1m54.313520609s) [noise=true , address=77.90.31.18:4835 , piece_id=W6FEWYN4QCZC5IQFUG2YXSGKHSKKFRGETXK76LFKAVACLA37H7PA, seg_pos={0 0}, upload=261114]: Uploading piece... Aug 19 04:28:13.672688665 (251h1m54.403956365s) [upload=261114]: Next returned nil segment Aug 19 04:28:13.672710426 (251h1m54.403978086s) [upload=261114]: Waiting for error group managing segments... Aug 19 04:28:13.681638350 (251h1m54.412906s) [noise=true , address=79.127.219.48:4085 , piece_id=AT2YETVN3DCMAPBL45XWQQCTW5FR6JC4PWN3CRIMZPQW5OG3RKKQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil> Aug 19 04:28:28.955286140 (251h2m9.68655377s) [noise=true , address=77.90.31.12:4596 , piece_id=AEOFPZ64F5FUTWHLMKFVKQGNGYJYOIU7OZPLYGS7ULR52F5XSNYQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1xgzDjjKfA3YT4tjGq6Esw9xfhA4xWMWnDbyAmNFUcP85qyFGf, address:77.90.31.12:4596): protocol: rpc: dial tcp 77.90.31.12:4596: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:28:45.285087265 (251h2m26.016354885s) [noise=true , address=104.36.82.60:4354 , piece_id=76HWEIKV5672IF34IRITARBMKGTN7OJU6GHQZZCDAP2ZUR2JNJFA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil> Aug 19 04:28:45.286367592 (251h2m26.017635222s) [noise=true , address=104.36.82.13:4204 , piece_id=LWFS4W2N7K6SWYKXYHZRJACAYYDBIB7OGVETZWKVTWWWJBZPS33A, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil> Aug 19 04:28:45.286429859 (251h2m26.017697489s) [noise=true , address=104.36.82.12:4156 , piece_id=RMBTQYXVS5NKE2ON3WW5T2CBE6S7S6T74VVVLMUSM5C44UXGRAYA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil> Aug 19 04:28:45.288231957 (251h2m26.019499578s) [noise=true , address=104.36.82.15:4289 , piece_id=GVBKZHWTPFDGYCM6U5EUYLBOTGCTTLPDWW2WZKZZEVNG4UZ3XEYA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil> Aug 19 04:28:45.316902561 (251h2m26.048170191s) [noise=true , address=104.36.82.44:4880 , piece_id=ZRDRV5AWRCYSUZJANVESK3XUCMLQQXF33WPV6QPX6HAZBYHXOUUQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil> Aug 19 04:28:46.178003461 (251h2m26.90927112s) [noise=true , address=104.36.82.5:4843 , piece_id=Q3SLO2FDTYYYYOQZARVS3SHAPM5J3GCLNFQ355QAV7U5MWKMZQQA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: <nil> Aug 19 04:29:13.506175578 (251h2m54.237443199s) [noise=true , address=45.91.116.6:4988 , piece_id=X36K53BYKX53WMZKQWN2FFKCKTHNLIY3KLLPXMQN2UBKZM2CTT6Q, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1TrcoCHhAJeEykP9hJNpygoPSdgFV5v6GzL48JpyNPiAj5ZJRS, address:45.91.116.6:4988): protocol: rpc: dial tcp 45.91.116.6:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506222607 (251h2m54.237490227s) [noise=true , address=45.91.116.9:4286 , piece_id=KUMIYTCCEU7XS2NMUTPXDSD7TMNASOBWGNWNDD7L6DMEAPXVIFWA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:19DfbHig9tzF4VRHtSWkExwUS2bdZ8GFgKnAD9wvMBG7qaesNa, address:45.91.116.9:4286): protocol: rpc: dial tcp 45.91.116.9:4286: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506254087 (251h2m54.237521717s) [noise=true , address=45.91.116.12:4243 , piece_id=GNLESUJ3FCKKUC7HOR7Z532H7LQT6ZOTOKLBGRN2R6UCJL4S3VFQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1e9VEMmhYq6PxqvTpxsoUHzqYVwujfj1UG7HSZFeiu8CderMpM, address:45.91.116.12:4243): protocol: rpc: dial tcp 45.91.116.12:4243: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506625304 (251h2m54.237892934s) [noise=true , address=77.90.31.21:4891 , piece_id=G57I4V2AOHDBI77GIIULZXTGCFC2BP6CQRX25KA7UNNIJ3WTAGPQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:17RDAerfx8AW5NsgWLTXLNDbuAEWQFT3SVM4iwdmuBzWrML83g, address:77.90.31.21:4891): protocol: rpc: dial tcp 77.90.31.21:4891: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506656563 (251h2m54.237924213s) [noise=true , address=77.90.31.19:4862 , piece_id=7CQOCX2FYKPQCTCBFMN7BMTD3FE2D2X4YIHFLCL22AOSYLKUO33A, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1Tvmo3nSgBPcFYW9MmZQGUeMoD3m3ykJHwVMj6tj69iQKgX3Mz, address:77.90.31.19:4862): protocol: rpc: dial tcp 77.90.31.19:4862: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 19 04:29:13.506689415 (251h2m54.237957045s) [noise=true , address=77.90.31.18:4835 , piece_id=W6FEWYN4QCZC5IQFUG2YXSGKHSKKFRGETXK76LFKAVACLA37H7PA, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:12hAQp2Zr7WSaQSTCZf5pwSyuBHHSrZWq5EVz5hFvSA5ddWebqg, address:77.90.31.18:4835): protocol: write tcp 5.161.89.252:27410->77.90.31.18:4835: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506740311 (251h2m54.23800793s) [noise=true , address=45.91.116.7:4037 , piece_id=B42RTCUIHRS5OUFJ5S5E6QWRKEN5H34IMBURUUSSUL53Y2FUYKZQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1iNPwE2XSfXcaXAXcrH2jcyNGYeDiiWvzAcDQffrmjPwTVqVmK, address:45.91.116.7:4037): protocol: rpc: dial tcp 45.91.116.7:4037: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506774885 (251h2m54.238042535s) [noise=true , address=77.90.31.17:4801 , piece_id=A4D6GQYALBOB46EE7D2KI6422NES4M43E4DTX4CKBLNNP2VHZUJQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:127QRLNwZLcR5gnAgZrCjSUsH5T4jESmZ4kWeAnpj9YxaTSjoYe, address:77.90.31.17:4801): protocol: write tcp 5.161.89.252:48394->77.90.31.17:4801: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506808788 (251h2m54.238076419s) [noise=true , address=45.91.116.4:4952 , piece_id=ZXPM75JCQGD3XQ3BTIP4O6SOLR6XCFNBUYQKKD56RTFVCMEGFWDQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1PcGuaN6uLz1HVzMgBp4vViKJw2KrmaFatL1w9DHgiYqccMUJ9, address:45.91.116.4:4952): protocol: write tcp 5.161.89.252:61300->45.91.116.4:4952: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506825761 (251h2m54.23809341s) [noise=true , address=77.90.31.13:4645 , piece_id=TGFZGBBRZLA32ZJS3BGT7OSHJW4S34PWMYSQKCZT4RG2IUTGPZSQ, seg_pos={0 0}, upload=261114]: Done uploading piece. err: ecclient: upload failed (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg, address:77.90.31.13:4645): protocol: rpc: dial tcp 77.90.31.13:4645: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 04:29:13.506906624 (251h2m54.238174313s) [seg_pos={0 0}, upload=261114]: Done waiting for segment. successful: 7 optimal: 14 errs: <nil> Aug 19 04:29:13.523807608 (251h2m54.255075238s) [upload=261114]: Done uploading segments... Aug 19 04:29:13.523824049 (251h2m54.255091679s) [upload=261114]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 7) Duration: 530.064792ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 201.24375ms FAIL (2025-08-19 04:58:24 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:26171->127.0.0.1:53: i/o timeout Duration: 201.261664ms edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 38.410354ms FAIL (2025-08-19 04:58:13 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:26171->127.0.0.1:53: i/o timeout Duration: 38.436041ms edge authservice http us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 390.906427ms FAIL (2025-08-19 04:58:11 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:26171->127.0.0.1:53: i/o timeout Duration: 390.926677ms edge gateway us-select-1
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 9.617656ms target: http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt FAIL (2025-08-19 04:58:09 UTC) orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp: lookup link.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:34965->127.0.0.1:53: i/o timeout Duration: 9.638306ms edge linksharing us-select-1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 578.698308ms FAIL (2025-08-19 04:58:08 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:34390->127.0.0.1:53: i/o timeout Duration: 578.714089ms edge authservice http bgp eu1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 4.797954091s FAIL (2025-08-19 04:58:07 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 4.797977314s core uplink ap1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1)

checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)