Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
157.34806ms
FAIL (2025-08-31 11:04:14 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 157.37465ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
160.730586ms
FAIL (2025-08-31 11:04:13 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 160.767155ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
4 out of 60 is failed (threshold=3) [..............XXXX..........................................] duration:
183.47596ms
FAIL (2025-08-31 11:04:12 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 183.496657ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
580.737606ms
FAIL (2025-08-31 11:04:08 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 100.100.100.100:53: read udp 100.64.0.137:62187->100.100.100.100:53: i/o timeout
Duration: 580.759847ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
5 out of 60 is failed (threshold=3) [...................XX.XXX...................................] duration:
524.885912ms
FAIL (2025-08-31 11:04:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
g 31 10:36:12.150915169 (44h27m47.503700297s) [upload=39171]: Done starting upload
Aug 31 10:36:12.151058448 (44h27m47.503843586s) [upload=39171]: Uploading segments...
Aug 31 10:36:12.151350147 (44h27m47.504135276s) [upload=39171]: Got next segment. Inline: false
Aug 31 10:36:12.151407877 (44h27m47.504193015s) [upload=39171]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 31 10:36:12.151414058 (44h27m47.504199186s) [upload=39171]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 31 10:36:12.181724654 (44h27m47.534509782s) [seg_pos={0 0}, upload=39171]: Begin upload segment...
Aug 31 10:36:12.181759048 (44h27m47.534544176s) [seg_pos={0 0}, upload=39171]: Done begin upload segment.
Aug 31 10:36:12.181787412 (44h27m47.534572551s) [noise=true , address=104.36.82.47:4543 , piece_id=OKISSUYRQP4GGIRDDPYFO6MCHDAFJEWCGHLI3P5T7EHBGOD7CYQA, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.181865228 (44h27m47.534650367s) [noise=true , address=45.91.116.12:4239 , piece_id=JIN2FAKX4JF3WVYMMICMIFQDIJT7ZZ4XCVCTDTKB7B3OFDTBY6HQ, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.182335613 (44h27m47.535120752s) [noise=true , address=77.90.31.10:4545 , piece_id=R5HG7NAZ2BFJCIYJOMV4BAD3Q7MAANXZJIFUDHBJ56PMITAB7FJA, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.182439810 (44h27m47.535224938s) [noise=true , address=104.36.82.11:4130 , piece_id=NEJRZC6MR7DTWS62XZZPJXTZW6GVHK4HIK6BXFD3QI46UGQACR3Q, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.182733102 (44h27m47.53551823s) [noise=true , address=45.91.116.7:4034 , piece_id=Z5BA6X346T3H2CITKWTLJDZOUOTVCHTXPUDPO2OKKXGRQF3YUMKQ, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.182828772 (44h27m47.535613901s) [noise=true , address=79.127.205.246:4280 , piece_id=BG3RNCBTBDNRY3MMJUQNKJP2RWH57UODQBKK66KK2WEDXA6FJQWA, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.183134137 (44h27m47.535919276s) [noise=true , address=104.36.82.48:4589 , piece_id=U4VSTDANJ6ZGA6PTOLZBKGBLD6K73SG4FQT6PUIH3QPWIO5A7YQQ, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.183571981 (44h27m47.53635711s) [noise=true , address=77.90.31.13:4629 , piece_id=746YIFKZ7XTJCLHAV2K2TWYRIJP5PFS47B5D37PFT6MERB62QM6Q, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.183905159 (44h27m47.536690288s) [noise=true , address=104.36.82.55:4212 , piece_id=BBTOTIURLHZXSY3NUBOJXHOB4U3HJIFEJM2AUO35DFACNAWBEZJA, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.184080729 (44h27m47.536865858s) [noise=true , address=45.91.116.13:4176 , piece_id=TTOEVZ6NGPERE27QNANZQVNP2QSCVGITHW5EFOV34EJ2KJ2GTVQA, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.184383600 (44h27m47.537168737s) [noise=true , address=77.90.31.17:4787 , piece_id=YLZWY3HQRSKHUHBOOVY36TATZ5IN7X2C27IXXCJKZZF4MHSRYH4A, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.184492785 (44h27m47.537277914s) [noise=true , address=77.90.31.14:4678 , piece_id=4XGVSFNNY6MGP5IOHJIBXL3S7JIPYAXKHFIAE6BSX5TAKTCJ4JVQ, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.184744750 (44h27m47.537529878s) [noise=true , address=45.91.116.9:4281 , piece_id=EWWXWQPBEDE724UFU2Z7YJ4E5DUI45OMNLHEFCC4OMG7ZQLWWBBQ, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.184850529 (44h27m47.537635657s) [noise=true , address=104.36.82.56:4264 , piece_id=S67WK5JPS46KPUHBQPLDDRGYJOLN6EEVOE5BFYT74U2IIXZFWXTA, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.185105669 (44h27m47.537890807s) [noise=true , address=77.90.31.21:4891 , piece_id=IKWWTL4THNTJGQ3CQLADO4KY2KFOXPASHSALNGE7NBUAXHPV3MBA, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.185259740 (44h27m47.538044868s) [noise=true , address=64.49.13.138:4482 , piece_id=JNYXEQ6PTZYSW5ULU55W5ZFEBEW25KXCDA36K47K4ADKODUZOKYA, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.185545127 (44h27m47.538330255s) [noise=true , address=104.36.82.23:4458 , piece_id=Y5IELCEOPOLHYYU2WEAZM7ZQMS3W6NR5YS64Y4UYD5MYZZQUJY4A, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.185657418 (44h27m47.538442556s) [noise=true , address=104.36.82.44:4883 , piece_id=455DKBRQXXYON4JM5CQ4SE5WR3WTTH3AJUAB6ZEHEHSE4RE6LP4A, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.220474138 (44h27m47.573259266s) [noise=true , address=77.90.31.21:4891 , piece_id=IKWWTL4THNTJGQ3CQLADO4KY2KFOXPASHSALNGE7NBUAXHPV3MBA, seg_pos={0 0}, upload=39171]: Done uploading piece. err: ecclient: upload failed (node:17RDAerfx8AW5NsgWLTXLNDbuAEWQFT3SVM4iwdmuBzWrML83g, address:77.90.31.21:4891): protocol: write tcp 5.161.89.252:27554->77.90.31.21:4891: write: no route to host; conn closed: op
Aug 31 10:36:12.302528983 (44h27m47.655314111s) [upload=39171]: Next returned nil segment
Aug 31 10:36:12.302541757 (44h27m47.655326885s) [upload=39171]: Waiting for error group managing segments...
Aug 31 10:36:12.316638798 (44h27m47.669423926s) [noise=true , address=104.36.82.56:4264 , piece_id=S67WK5JPS46KPUHBQPLDDRGYJOLN6EEVOE5BFYT74U2IIXZFWXTA, seg_pos={0 0}, upload=39171]: Done uploading piece. err: <nil>
Aug 31 10:36:12.316773852 (44h27m47.66955899s) [noise=true , address=104.36.82.44:4883 , piece_id=455DKBRQXXYON4JM5CQ4SE5WR3WTTH3AJUAB6ZEHEHSE4RE6LP4A, seg_pos={0 0}, upload=39171]: Done uploading piece. err: <nil>
Aug 31 10:36:12.317388379 (44h27m47.670173506s) [noise=true , address=104.36.82.47:4543 , piece_id=OKISSUYRQP4GGIRDDPYFO6MCHDAFJEWCGHLI3P5T7EHBGOD7CYQA, seg_pos={0 0}, upload=39171]: Done uploading piece. err: <nil>
Aug 31 10:36:12.317831102 (44h27m47.67061623s) [noise=true , address=104.36.82.23:4458 , piece_id=Y5IELCEOPOLHYYU2WEAZM7ZQMS3W6NR5YS64Y4UYD5MYZZQUJY4A, seg_pos={0 0}, upload=39171]: Done uploading piece. err: <nil>
Aug 31 10:36:12.317981115 (44h27m47.670766243s) [noise=true , address=104.36.82.55:4212 , piece_id=BBTOTIURLHZXSY3NUBOJXHOB4U3HJIFEJM2AUO35DFACNAWBEZJA, seg_pos={0 0}, upload=39171]: Done uploading piece. err: <nil>
Aug 31 10:36:12.318644403 (44h27m47.671429531s) [noise=true , address=104.36.82.11:4130 , piece_id=NEJRZC6MR7DTWS62XZZPJXTZW6GVHK4HIK6BXFD3QI46UGQACR3Q, seg_pos={0 0}, upload=39171]: Done uploading piece. err: <nil>
Aug 31 10:36:12.319463665 (44h27m47.672248794s) [noise=true , address=104.36.82.48:4589 , piece_id=U4VSTDANJ6ZGA6PTOLZBKGBLD6K73SG4FQT6PUIH3QPWIO5A7YQQ, seg_pos={0 0}, upload=39171]: Done uploading piece. err: <nil>
Aug 31 10:36:12.365155230 (44h27m47.717940358s) [noise=true , address=45.91.116.13:4176 , piece_id=TTOEVZ6NGPERE27QNANZQVNP2QSCVGITHW5EFOV34EJ2KJ2GTVQA, seg_pos={0 0}, upload=39171]: Done uploading piece. err: <nil>
Aug 31 10:36:12.382100673 (44h27m47.734885801s) [noise=true , address=45.91.116.9:4281 , piece_id=EWWXWQPBEDE724UFU2Z7YJ4E5DUI45OMNLHEFCC4OMG7ZQLWWBBQ, seg_pos={0 0}, upload=39171]: Done uploading piece. err: <nil>
Aug 31 10:36:12.382477341 (44h27m47.73526248s) [noise=true , address=45.91.116.12:4239 , piece_id=JIN2FAKX4JF3WVYMMICMIFQDIJT7ZZ4XCVCTDTKB7B3OFDTBY6HQ, seg_pos={0 0}, upload=39171]: Done uploading piece. err: <nil>
Aug 31 10:36:12.385851305 (44h27m47.738636442s) [noise=true , address=45.91.116.7:4034 , piece_id=Z5BA6X346T3H2CITKWTLJDZOUOTVCHTXPUDPO2OKKXGRQF3YUMKQ, seg_pos={0 0}, upload=39171]: Done uploading piece. err: <nil>
Aug 31 10:36:12.670547674 (44h27m48.023332772s) [noise=true , address=79.127.205.246:4280 , piece_id=BG3RNCBTBDNRY3MMJUQNKJP2RWH57UODQBKK66KK2WEDXA6FJQWA, seg_pos={0 0}, upload=39171]: Done uploading piece. err: <nil>
Aug 31 10:36:12.734232925 (44h27m48.087018034s) [noise=true , address=64.49.13.138:4482 , piece_id=JNYXEQ6PTZYSW5ULU55W5ZFEBEW25KXCDA36K47K4ADKODUZOKYA, seg_pos={0 0}, upload=39171]: Done uploading piece. err: <nil>
Aug 31 10:37:12.136109118 (44h28m47.488894215s) [noise=true , address=77.90.31.17:4787 , piece_id=YLZWY3HQRSKHUHBOOVY36TATZ5IN7X2C27IXXCJKZZF4MHSRYH4A, seg_pos={0 0}, upload=39171]: Done uploading piece. err: ecclient: upload failed (node:1Lietonb5aVbaZQt6wQ2EoxwJkSEn9oL9Dwyrc9mTyomEKikNt, address:77.90.31.17:4787): protocol: rpc: dial tcp 77.90.31.17:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 31 10:37:12.136180632 (44h28m47.48896573s) [noise=true , address=77.90.31.10:4545 , piece_id=R5HG7NAZ2BFJCIYJOMV4BAD3Q7MAANXZJIFUDHBJ56PMITAB7FJA, seg_pos={0 0}, upload=39171]: Done uploading piece. err: ecclient: upload failed (node:1ZFFijCUFET1ShEJuav6uAzFhfraCEcr6seuySYN9ipn4wsZGX, address:77.90.31.10:4545): protocol: rpc: dial tcp 77.90.31.10:4545: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 31 10:37:12.136217150 (44h28m47.489002248s) [noise=true , address=77.90.31.13:4629 , piece_id=746YIFKZ7XTJCLHAV2K2TWYRIJP5PFS47B5D37PFT6MERB62QM6Q, seg_pos={0 0}, upload=39171]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: rpc: dial tcp 77.90.31.13:4629: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 31 10:37:12.136244112 (44h28m47.48902921s) [noise=true , address=77.90.31.14:4678 , piece_id=4XGVSFNNY6MGP5IOHJIBXL3S7JIPYAXKHFIAE6BSX5TAKTCJ4JVQ, seg_pos={0 0}, upload=39171]: Done uploading piece. err: ecclient: upload failed (node:12beu3SUZMPNWfbeHV4tLtx9UVZoEHh4Kt7oBhT59w3bGBE1ovP, address:77.90.31.14:4678): protocol: rpc: dial tcp 77.90.31.14:4678: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 31 10:37:12.136292463 (44h28m47.48907756s) [seg_pos={0 0}, upload=39171]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Aug 31 10:37:12.152297525 (44h28m47.505082633s) [upload=39171]: Done uploading segments...
Aug 31 10:37:12.152314989 (44h28m47.505100086s) [upload=39171]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 524.912713ms
core
uplink
us-select-1
us1-placement-20
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
2 out of 60 is failed (threshold=3) [..................XX........................................] duration:
162.650773ms
FAIL (2025-08-31 11:04:03 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 100.100.100.100:53: read udp 100.64.0.137:62187->100.100.100.100:53: i/o timeout
Duration: 162.676651ms
edge
authservice
http
bgp
us1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
4.73725655s
FAIL (2025-08-31 11:04:02 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 4.737284522s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...................XX.......................................] duration:
2.942762833s
FAIL (2025-08-31 11:04:00 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.942783254s
core
uplink
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
250.42785ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011
FAIL (2025-08-31 11:03:59 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": dial tcp: lookup uptime-checks.gateway.storjshare.io on 100.100.100.100:53: read udp 100.64.0.137:29236->100.100.100.100:53: i/o timeout
Duration: 250.458408ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [................XX..........................................] duration:
275.394134ms target:
https://eu1.storj.io/login
FAIL (2025-08-31 11:03:56 UTC)
orbiter warning: Get "https://eu1.storj.io/login": dial tcp 35.207.121.91:443: i/o timeout
Duration: 275.416807ms
core
satellite
eu1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
4 out of 60 is failed (threshold=3) [...............XXXX.........................................] duration:
286.304105ms
FAIL (2025-08-31 11:03:54 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 286.320055ms
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
732.174064ms target:
https://ap1.storj.io/login
FAIL (2025-08-31 11:03:53 UTC)
orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.157.232:443: i/o timeout
Duration: 732.196807ms
core
satellite
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
2.205051093s
FAIL (2025-08-31 11:03:52 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 2.205078064s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing)antiblip:
4 out of 60 is failed (threshold=3) [.................XXXX.......................................] duration:
28.712115ms target:
http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt
FAIL (2025-08-31 11:03:47 UTC)
orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout
Duration: 28.73076ms
edge
linksharing
us-select-1
|
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)antiblip:
3 out of 15 is failed (threshold=6) [............XXX] duration:
56.031419ms remote:
"tcp:54.243.86.28:443" response headers:
X-Lambda-Id=c6207948-cc73-40b2-9a2a-e948767f3d5e Vary=Accept-Encoding Cf-Cache-Status=HIT Last-Modified=Tue, 26 Aug 2025 19:05:15 GMT Alt-Svc=h3=":443"; ma=86400 Age=403111 Surrogate-Key=www.storj.io 65d78d3b8dbb1f54a94020c7 pageId:67eb0cf63d37c62ac04e0ff0 Set-Cookie=_cfuvid=BS.AKbQmrk4hqFY2oA.yz3IcXXB2mvNzp5C5hBcxPBc-1756638226260-0.0.1.1-604800000; path=/; domain=.cdn.webflow.com; HttpOnly; Secure; SameSite=None X-Cluster-Name=us-east-1-prod-hosting-red Date=Sun, 31 Aug 2025 11:03:46 GMT Content-Type=text/html Cf-Ray=977c0b11eb04bc3b-IAD Surrogate-Control=max-age=2147483647 target:
https://www.storj.io/
FAIL (2025-08-31 11:03:46 UTC)
orbiter warning: no valid certificate chain appears to exist after 336h0m0s
Duration: 56.054502ms
core
website
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
154.488086ms
FAIL (2025-08-31 11:03:44 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 154.515787ms
core
uplink
us1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [.........X..................................................] duration:
120.573204ms target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-08-31 11:03:42 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp: lookup link.storjshare.io on 100.100.100.100:53: read udp 100.64.0.137:51715->100.100.100.100:53: i/o timeout
Duration: 120.591529ms
edge
linksharing
bgp
eu1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
3 out of 60 is failed (threshold=3) [...................XXX......................................] duration:
40.613622ms
FAIL (2025-08-31 11:03:41 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 40.632757ms
edge
authservice
http
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
966.928706ms
FAIL (2025-08-31 11:03:41 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 100.100.100.100:53: read udp 100.64.0.137:62187->100.100.100.100:53: i/o timeout
Duration: 966.954174ms
edge
authservice
http
bgp
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
1.984933145s
FAIL (2025-08-31 11:03:40 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 1.984985694s
core
uplink
ap1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.............XXX............................................] duration:
368.444502ms
FAIL (2025-08-31 11:03:38 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 368.465241ms
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [.................XX.........................................] duration:
279.763ms target:
https://eu1.storj.io/login
FAIL (2025-08-31 11:03:36 UTC)
orbiter warning: Get "https://eu1.storj.io/login": dial tcp 35.207.121.91:443: i/o timeout
Duration: 279.783499ms
core
satellite
eu1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
4 out of 60 is failed (threshold=3) [................XXXX........................................] duration:
406.77204ms
FAIL (2025-08-31 11:03:36 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 406.791728ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)antiblip:
2 out of 60 is failed (threshold=3) [...................XX.......................................] duration:
859.373354ms
FAIL (2025-08-31 11:03:35 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 859.401158ms
core
uplink
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
249.667221ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011
FAIL (2025-08-31 11:03:32 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": dial tcp: lookup uptime-checks.gateway.storjshare.io on 100.100.100.100:53: read udp 100.64.0.137:29236->100.100.100.100:53: i/o timeout
Duration: 249.69279ms
edge
gateway
vhost
bgp
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
2 out of 60 is failed (threshold=3) [....................XX......................................] duration:
3.057904034s
FAIL (2025-08-31 11:03:28 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 3.057956082s
core
uplink
eu1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
1.962158641s
FAIL (2025-08-31 11:03:23 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 1.96217904s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
2.166473341s
FAIL (2025-08-31 11:03:19 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 2.166497405s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
4.810162048s
FAIL (2025-08-31 11:03:18 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 4.810181183s
core
uplink
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
65.290752ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-08-31 11:03:17 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp: lookup uptime-checks.gateway.storjshare.io on 100.100.100.100:53: read udp 100.64.0.137:29236->100.100.100.100:53: i/o timeout
Duration: 65.303747ms
edge
gateway
vhost
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
3 out of 60 is failed (threshold=3) [....................XXX.....................................] duration:
36.034713ms
FAIL (2025-08-31 11:03:16 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 36.060421ms
edge
authservice
http
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
698.110794ms
FAIL (2025-08-31 11:03:15 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 100.100.100.100:53: read udp 100.64.0.137:62187->100.100.100.100:53: i/o timeout
Duration: 698.127376ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
4 out of 60 is failed (threshold=3) [.................XXXX.......................................] duration:
170.340649ms
FAIL (2025-08-31 11:03:13 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 170.36235ms
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
4 out of 60 is failed (threshold=3) [................XXXX........................................] duration:
23.107145ms target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-31 11:03:12 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp 45.38.24.2:443: i/o timeout
Duration: 23.149434ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing)antiblip:
4 out of 60 is failed (threshold=3) [..................XXXX......................................] duration:
5.357635ms target:
http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt
FAIL (2025-08-31 11:03:10 UTC)
orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout
Duration: 5.390819ms
edge
linksharing
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
5 out of 60 is failed (threshold=3) [....................XX.XXX..................................] duration:
513.08192ms
FAIL (2025-08-31 11:03:09 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
g 31 10:36:12.150915169 (44h27m47.503700297s) [upload=39171]: Done starting upload
Aug 31 10:36:12.151058448 (44h27m47.503843586s) [upload=39171]: Uploading segments...
Aug 31 10:36:12.151350147 (44h27m47.504135276s) [upload=39171]: Got next segment. Inline: false
Aug 31 10:36:12.151407877 (44h27m47.504193015s) [upload=39171]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 31 10:36:12.151414058 (44h27m47.504199186s) [upload=39171]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 31 10:36:12.181724654 (44h27m47.534509782s) [seg_pos={0 0}, upload=39171]: Begin upload segment...
Aug 31 10:36:12.181759048 (44h27m47.534544176s) [seg_pos={0 0}, upload=39171]: Done begin upload segment.
Aug 31 10:36:12.181787412 (44h27m47.534572551s) [noise=true , address=104.36.82.47:4543 , piece_id=OKISSUYRQP4GGIRDDPYFO6MCHDAFJEWCGHLI3P5T7EHBGOD7CYQA, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.181865228 (44h27m47.534650367s) [noise=true , address=45.91.116.12:4239 , piece_id=JIN2FAKX4JF3WVYMMICMIFQDIJT7ZZ4XCVCTDTKB7B3OFDTBY6HQ, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.182335613 (44h27m47.535120752s) [noise=true , address=77.90.31.10:4545 , piece_id=R5HG7NAZ2BFJCIYJOMV4BAD3Q7MAANXZJIFUDHBJ56PMITAB7FJA, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.182439810 (44h27m47.535224938s) [noise=true , address=104.36.82.11:4130 , piece_id=NEJRZC6MR7DTWS62XZZPJXTZW6GVHK4HIK6BXFD3QI46UGQACR3Q, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.182733102 (44h27m47.53551823s) [noise=true , address=45.91.116.7:4034 , piece_id=Z5BA6X346T3H2CITKWTLJDZOUOTVCHTXPUDPO2OKKXGRQF3YUMKQ, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.182828772 (44h27m47.535613901s) [noise=true , address=79.127.205.246:4280 , piece_id=BG3RNCBTBDNRY3MMJUQNKJP2RWH57UODQBKK66KK2WEDXA6FJQWA, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.183134137 (44h27m47.535919276s) [noise=true , address=104.36.82.48:4589 , piece_id=U4VSTDANJ6ZGA6PTOLZBKGBLD6K73SG4FQT6PUIH3QPWIO5A7YQQ, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.183571981 (44h27m47.53635711s) [noise=true , address=77.90.31.13:4629 , piece_id=746YIFKZ7XTJCLHAV2K2TWYRIJP5PFS47B5D37PFT6MERB62QM6Q, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.183905159 (44h27m47.536690288s) [noise=true , address=104.36.82.55:4212 , piece_id=BBTOTIURLHZXSY3NUBOJXHOB4U3HJIFEJM2AUO35DFACNAWBEZJA, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.184080729 (44h27m47.536865858s) [noise=true , address=45.91.116.13:4176 , piece_id=TTOEVZ6NGPERE27QNANZQVNP2QSCVGITHW5EFOV34EJ2KJ2GTVQA, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.184383600 (44h27m47.537168737s) [noise=true , address=77.90.31.17:4787 , piece_id=YLZWY3HQRSKHUHBOOVY36TATZ5IN7X2C27IXXCJKZZF4MHSRYH4A, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.184492785 (44h27m47.537277914s) [noise=true , address=77.90.31.14:4678 , piece_id=4XGVSFNNY6MGP5IOHJIBXL3S7JIPYAXKHFIAE6BSX5TAKTCJ4JVQ, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.184744750 (44h27m47.537529878s) [noise=true , address=45.91.116.9:4281 , piece_id=EWWXWQPBEDE724UFU2Z7YJ4E5DUI45OMNLHEFCC4OMG7ZQLWWBBQ, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.184850529 (44h27m47.537635657s) [noise=true , address=104.36.82.56:4264 , piece_id=S67WK5JPS46KPUHBQPLDDRGYJOLN6EEVOE5BFYT74U2IIXZFWXTA, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.185105669 (44h27m47.537890807s) [noise=true , address=77.90.31.21:4891 , piece_id=IKWWTL4THNTJGQ3CQLADO4KY2KFOXPASHSALNGE7NBUAXHPV3MBA, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.185259740 (44h27m47.538044868s) [noise=true , address=64.49.13.138:4482 , piece_id=JNYXEQ6PTZYSW5ULU55W5ZFEBEW25KXCDA36K47K4ADKODUZOKYA, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.185545127 (44h27m47.538330255s) [noise=true , address=104.36.82.23:4458 , piece_id=Y5IELCEOPOLHYYU2WEAZM7ZQMS3W6NR5YS64Y4UYD5MYZZQUJY4A, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.185657418 (44h27m47.538442556s) [noise=true , address=104.36.82.44:4883 , piece_id=455DKBRQXXYON4JM5CQ4SE5WR3WTTH3AJUAB6ZEHEHSE4RE6LP4A, seg_pos={0 0}, upload=39171]: Uploading piece...
Aug 31 10:36:12.220474138 (44h27m47.573259266s) [noise=true , address=77.90.31.21:4891 , piece_id=IKWWTL4THNTJGQ3CQLADO4KY2KFOXPASHSALNGE7NBUAXHPV3MBA, seg_pos={0 0}, upload=39171]: Done uploading piece. err: ecclient: upload failed (node:17RDAerfx8AW5NsgWLTXLNDbuAEWQFT3SVM4iwdmuBzWrML83g, address:77.90.31.21:4891): protocol: write tcp 5.161.89.252:27554->77.90.31.21:4891: write: no route to host; conn closed: op
Aug 31 10:36:12.302528983 (44h27m47.655314111s) [upload=39171]: Next returned nil segment
Aug 31 10:36:12.302541757 (44h27m47.655326885s) [upload=39171]: Waiting for error group managing segments...
Aug 31 10:36:12.316638798 (44h27m47.669423926s) [noise=true , address=104.36.82.56:4264 , piece_id=S67WK5JPS46KPUHBQPLDDRGYJOLN6EEVOE5BFYT74U2IIXZFWXTA, seg_pos={0 0}, upload=39171]: Done uploading piece. err: <nil>
Aug 31 10:36:12.316773852 (44h27m47.66955899s) [noise=true , address=104.36.82.44:4883 , piece_id=455DKBRQXXYON4JM5CQ4SE5WR3WTTH3AJUAB6ZEHEHSE4RE6LP4A, seg_pos={0 0}, upload=39171]: Done uploading piece. err: <nil>
Aug 31 10:36:12.317388379 (44h27m47.670173506s) [noise=true , address=104.36.82.47:4543 , piece_id=OKISSUYRQP4GGIRDDPYFO6MCHDAFJEWCGHLI3P5T7EHBGOD7CYQA, seg_pos={0 0}, upload=39171]: Done uploading piece. err: <nil>
Aug 31 10:36:12.317831102 (44h27m47.67061623s) [noise=true , address=104.36.82.23:4458 , piece_id=Y5IELCEOPOLHYYU2WEAZM7ZQMS3W6NR5YS64Y4UYD5MYZZQUJY4A, seg_pos={0 0}, upload=39171]: Done uploading piece. err: <nil>
Aug 31 10:36:12.317981115 (44h27m47.670766243s) [noise=true , address=104.36.82.55:4212 , piece_id=BBTOTIURLHZXSY3NUBOJXHOB4U3HJIFEJM2AUO35DFACNAWBEZJA, seg_pos={0 0}, upload=39171]: Done uploading piece. err: <nil>
Aug 31 10:36:12.318644403 (44h27m47.671429531s) [noise=true , address=104.36.82.11:4130 , piece_id=NEJRZC6MR7DTWS62XZZPJXTZW6GVHK4HIK6BXFD3QI46UGQACR3Q, seg_pos={0 0}, upload=39171]: Done uploading piece. err: <nil>
Aug 31 10:36:12.319463665 (44h27m47.672248794s) [noise=true , address=104.36.82.48:4589 , piece_id=U4VSTDANJ6ZGA6PTOLZBKGBLD6K73SG4FQT6PUIH3QPWIO5A7YQQ, seg_pos={0 0}, upload=39171]: Done uploading piece. err: <nil>
Aug 31 10:36:12.365155230 (44h27m47.717940358s) [noise=true , address=45.91.116.13:4176 , piece_id=TTOEVZ6NGPERE27QNANZQVNP2QSCVGITHW5EFOV34EJ2KJ2GTVQA, seg_pos={0 0}, upload=39171]: Done uploading piece. err: <nil>
Aug 31 10:36:12.382100673 (44h27m47.734885801s) [noise=true , address=45.91.116.9:4281 , piece_id=EWWXWQPBEDE724UFU2Z7YJ4E5DUI45OMNLHEFCC4OMG7ZQLWWBBQ, seg_pos={0 0}, upload=39171]: Done uploading piece. err: <nil>
Aug 31 10:36:12.382477341 (44h27m47.73526248s) [noise=true , address=45.91.116.12:4239 , piece_id=JIN2FAKX4JF3WVYMMICMIFQDIJT7ZZ4XCVCTDTKB7B3OFDTBY6HQ, seg_pos={0 0}, upload=39171]: Done uploading piece. err: <nil>
Aug 31 10:36:12.385851305 (44h27m47.738636442s) [noise=true , address=45.91.116.7:4034 , piece_id=Z5BA6X346T3H2CITKWTLJDZOUOTVCHTXPUDPO2OKKXGRQF3YUMKQ, seg_pos={0 0}, upload=39171]: Done uploading piece. err: <nil>
Aug 31 10:36:12.670547674 (44h27m48.023332772s) [noise=true , address=79.127.205.246:4280 , piece_id=BG3RNCBTBDNRY3MMJUQNKJP2RWH57UODQBKK66KK2WEDXA6FJQWA, seg_pos={0 0}, upload=39171]: Done uploading piece. err: <nil>
Aug 31 10:36:12.734232925 (44h27m48.087018034s) [noise=true , address=64.49.13.138:4482 , piece_id=JNYXEQ6PTZYSW5ULU55W5ZFEBEW25KXCDA36K47K4ADKODUZOKYA, seg_pos={0 0}, upload=39171]: Done uploading piece. err: <nil>
Aug 31 10:37:12.136109118 (44h28m47.488894215s) [noise=true , address=77.90.31.17:4787 , piece_id=YLZWY3HQRSKHUHBOOVY36TATZ5IN7X2C27IXXCJKZZF4MHSRYH4A, seg_pos={0 0}, upload=39171]: Done uploading piece. err: ecclient: upload failed (node:1Lietonb5aVbaZQt6wQ2EoxwJkSEn9oL9Dwyrc9mTyomEKikNt, address:77.90.31.17:4787): protocol: rpc: dial tcp 77.90.31.17:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 31 10:37:12.136180632 (44h28m47.48896573s) [noise=true , address=77.90.31.10:4545 , piece_id=R5HG7NAZ2BFJCIYJOMV4BAD3Q7MAANXZJIFUDHBJ56PMITAB7FJA, seg_pos={0 0}, upload=39171]: Done uploading piece. err: ecclient: upload failed (node:1ZFFijCUFET1ShEJuav6uAzFhfraCEcr6seuySYN9ipn4wsZGX, address:77.90.31.10:4545): protocol: rpc: dial tcp 77.90.31.10:4545: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 31 10:37:12.136217150 (44h28m47.489002248s) [noise=true , address=77.90.31.13:4629 , piece_id=746YIFKZ7XTJCLHAV2K2TWYRIJP5PFS47B5D37PFT6MERB62QM6Q, seg_pos={0 0}, upload=39171]: Done uploading piece. err: ecclient: upload failed (node:12GZtxLCqqDT4EXE2oCWjynZwboY9wBq4BtbkisCeTdCcCncT9f, address:77.90.31.13:4629): protocol: rpc: dial tcp 77.90.31.13:4629: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 31 10:37:12.136244112 (44h28m47.48902921s) [noise=true , address=77.90.31.14:4678 , piece_id=4XGVSFNNY6MGP5IOHJIBXL3S7JIPYAXKHFIAE6BSX5TAKTCJ4JVQ, seg_pos={0 0}, upload=39171]: Done uploading piece. err: ecclient: upload failed (node:12beu3SUZMPNWfbeHV4tLtx9UVZoEHh4Kt7oBhT59w3bGBE1ovP, address:77.90.31.14:4678): protocol: rpc: dial tcp 77.90.31.14:4678: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 31 10:37:12.136292463 (44h28m47.48907756s) [seg_pos={0 0}, upload=39171]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Aug 31 10:37:12.152297525 (44h28m47.505082633s) [upload=39171]: Done uploading segments...
Aug 31 10:37:12.152314989 (44h28m47.505100086s) [upload=39171]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 513.11977ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
6 out of 60 is failed (threshold=3) [................XXXXXX......................................] duration:
514.927466ms
FAIL (2025-08-31 11:03:08 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 10)
======LOGS======
Aug 31 10:35:45.949357429 (44h27m21.302142558s) [upload=39166]: Starting upload
Aug 31 10:35:45.949453992 (44h27m21.302239119s) [upload=39166]: Done starting upload
Aug 31 10:35:45.949561915 (44h27m21.302347043s) [upload=39166]: Uploading segments...
Aug 31 10:35:45.949839758 (44h27m21.302624886s) [upload=39166]: Got next segment. Inline: false
Aug 31 10:35:45.949875034 (44h27m21.302660163s) [upload=39166]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 31 10:35:45.949880324 (44h27m21.302665453s) [upload=39166]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 31 10:35:45.974500057 (44h27m21.327285184s) [seg_pos={0 0}, upload=39166]: Begin upload segment...
Aug 31 10:35:45.974534781 (44h27m21.32731991s) [seg_pos={0 0}, upload=39166]: Done begin upload segment.
Aug 31 10:35:45.974561612 (44h27m21.32734674s) [noise=true , address=45.91.116.7:4010 , piece_id=P76FQJ7UMNJZZOVAED6OYJ4KTB67KHOZYJ7X4KYQLMZAVDYHPY2Q, seg_pos={0 0}, upload=39166]: Uploading piece...
Aug 31 10:35:45.974622567 (44h27m21.327407705s) [noise=true , address=77.90.31.17:4772 , piece_id=EXYIUPQYWHMWJX6YDTK7NMANOLDI6UPQK3SYUKPH4N42MED2CWYA, seg_pos={0 0}, upload=39166]: Uploading piece...
Aug 31 10:35:45.975048299 (44h27m21.327833426s) [noise=true , address=45.91.116.11:4117 , piece_id=Q4W2NO374GGMKEUS4HFVH2U2PPRCBS3BAUKKKU2KEESFQNVB5F6Q, seg_pos={0 0}, upload=39166]: Uploading piece...
Aug 31 10:35:45.975060932 (44h27m21.327846061s) [noise=true , address=77.90.31.16:4735 , piece_id=VT2UI46533CVPZLAMRC5TNZKQCCUWUWRBFXUUDZH7NWYVBYQXBJA, seg_pos={0 0}, upload=39166]: Uploading piece...
Aug 31 10:35:45.975453572 (44h27m21.3282387s) [noise=true , address=45.91.116.8:4047 , piece_id=PRK5W24CWG4TY3Q5D4YUPCKXXQ5KDGMI25L3DZ6Z3SPWIPV2OGZA, seg_pos={0 0}, upload=39166]: Uploading piece...
Aug 31 10:35:45.975479350 (44h27m21.328264488s) [noise=true , address=77.90.31.12:4590 , piece_id=ROSYARSOOVQHYE7VI4QQFXVTRWHXMZCVR2JS4WG2PTUBSXJDANGQ, seg_pos={0 0}, upload=39166]: Uploading piece...
Aug 31 10:35:45.975834078 (44h27m21.328619207s) [noise=true , address=77.90.31.10:4517 , piece_id=ETZJZFDL3HNZA3GSJS4GE2QZZCLSVYBZEDD2CWZIARBQ7XBVKRHQ, seg_pos={0 0}, upload=39166]: Uploading piece...
Aug 31 10:35:45.975889633 (44h27m21.328674761s) [noise=true , address=104.36.82.2:4730 , piece_id=NMRNV2ZOG7JDLKSYHNAG7FFLUR7MECSBEZK57FVZ2FKJCX6BHM2Q, seg_pos={0 0}, upload=39166]: Uploading piece...
Aug 31 10:35:45.976450218 (44h27m21.329235346s) [noise=true , address=77.90.31.14:4662 , piece_id=5GG7POTFX2VDBHTSDLLRARV5OQ5SNWUXASAFC734IGNLJIOR34EQ, seg_pos={0 0}, upload=39166]: Uploading piece...
Aug 31 10:35:45.976462792 (44h27m21.32924792s) [noise=true , address=45.91.116.4:4940 , piece_id=2LEW4ZRRSCYGNX2MZCUGZJPUSNSD7ZHI63WM7FNIODHGP5K5J2HA, seg_pos={0 0}, upload=39166]: Uploading piece...
Aug 31 10:35:45.976837658 (44h27m21.329622786s) [noise=true , address=45.91.116.6:4973 , piece_id=VGG3ORU6E2PDMAGENBR2AJWRMQ22QLAFI5DAQUB4XMO4KI5G4TQQ, seg_pos={0 0}, upload=39166]: Uploading piece...
Aug 31 10:35:45.976933107 (44h27m21.329718236s) [noise=true , address=104.36.82.15:4259 , piece_id=7N4OLUFMAZPMGDZCBPQWA5RI3QU4KTD4K53FXRBSAA6RDBKZLBVA, seg_pos={0 0}, upload=39166]: Uploading piece...
Aug 31 10:35:45.977198717 (44h27m21.329983845s) [noise=true , address=104.36.82.5:4840 , piece_id=VOFINL532TIV77L5WVMMLOU6ABJBDYYUNX7U4XSBSFJE3JQICJQQ, seg_pos={0 0}, upload=39166]: Uploading piece...
Aug 31 10:35:45.977335244 (44h27m21.330120372s) [noise=true , address=45.91.116.10:4081 , piece_id=2UPUNDEYUJXFD44M62CXSKFNFGTVY2F5UKYZOVXHMZKIHTFB6DUQ, seg_pos={0 0}, upload=39166]: Uploading piece...
Aug 31 10:35:45.977693609 (44h27m21.330478747s) [noise=true , address=77.90.31.17:4770 , piece_id=GMBH47VTWWVOFSYF5HSN2FTTQZS6MUYJZ2C3243INM4TKE3F4ZJA, seg_pos={0 0}, upload=39166]: Uploading piece...
Aug 31 10:35:45.978139749 (44h27m21.330924877s) [noise=true , address=45.91.116.9:4262 , piece_id=2DTFYRZDKYB73XUA2QXKATIABE522EIFS2V5IGT3DPY6227F4Q4A, seg_pos={0 0}, upload=39166]: Uploading piece...
Aug 31 10:35:46.010680273 (44h27m21.363465391s) [noise=true , address=77.90.31.17:4770 , piece_id=GMBH47VTWWVOFSYF5HSN2FTTQZS6MUYJZ2C3243INM4TKE3F4ZJA, seg_pos={0 0}, upload=39166]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 5.161.89.252:33308->77.90.31.17:4770: write: no route to host; manager closed: conn closed: op
Aug 31 10:35:46.094242937 (44h27m21.447028065s) [upload=39166]: Next returned nil segment
Aug 31 10:35:46.094257233 (44h27m21.447042362s) [upload=39166]: Waiting for error group managing segments...
Aug 31 10:35:46.107643376 (44h27m21.460428494s) [noise=true , address=104.36.82.15:4259 , piece_id=7N4OLUFMAZPMGDZCBPQWA5RI3QU4KTD4K53FXRBSAA6RDBKZLBVA, seg_pos={0 0}, upload=39166]: Done uploading piece. err: <nil>
Aug 31 10:35:46.108127397 (44h27m21.460912515s) [noise=true , address=104.36.82.2:4730 , piece_id=NMRNV2ZOG7JDLKSYHNAG7FFLUR7MECSBEZK57FVZ2FKJCX6BHM2Q, seg_pos={0 0}, upload=39166]: Done uploading piece. err: <nil>
Aug 31 10:35:46.109366731 (44h27m21.462151858s) [noise=true , address=104.36.82.5:4840 , piece_id=VOFINL532TIV77L5WVMMLOU6ABJBDYYUNX7U4XSBSFJE3JQICJQQ, seg_pos={0 0}, upload=39166]: Done uploading piece. err: <nil>
Aug 31 10:35:46.158408011 (44h27m21.51119313s) [noise=true , address=45.91.116.11:4117 , piece_id=Q4W2NO374GGMKEUS4HFVH2U2PPRCBS3BAUKKKU2KEESFQNVB5F6Q, seg_pos={0 0}, upload=39166]: Done uploading piece. err: <nil>
Aug 31 10:35:46.160289913 (44h27m21.513075032s) [noise=true , address=45.91.116.9:4262 , piece_id=2DTFYRZDKYB73XUA2QXKATIABE522EIFS2V5IGT3DPY6227F4Q4A, seg_pos={0 0}, upload=39166]: Done uploading piece. err: <nil>
Aug 31 10:35:46.175403087 (44h27m21.528188235s) [noise=true , address=45.91.116.4:4940 , piece_id=2LEW4ZRRSCYGNX2MZCUGZJPUSNSD7ZHI63WM7FNIODHGP5K5J2HA, seg_pos={0 0}, upload=39166]: Done uploading piece. err: <nil>
Aug 31 10:35:46.180837789 (44h27m21.533622927s) [noise=true , address=45.91.116.7:4010 , piece_id=P76FQJ7UMNJZZOVAED6OYJ4KTB67KHOZYJ7X4KYQLMZAVDYHPY2Q, seg_pos={0 0}, upload=39166]: Done uploading piece. err: <nil>
Aug 31 10:35:46.191657048 (44h27m21.544442177s) [noise=true , address=45.91.116.8:4047 , piece_id=PRK5W24CWG4TY3Q5D4YUPCKXXQ5KDGMI25L3DZ6Z3SPWIPV2OGZA, seg_pos={0 0}, upload=39166]: Done uploading piece. err: <nil>
Aug 31 10:35:46.201182052 (44h27m21.55396718s) [noise=true , address=45.91.116.10:4081 , piece_id=2UPUNDEYUJXFD44M62CXSKFNFGTVY2F5UKYZOVXHMZKIHTFB6DUQ, seg_pos={0 0}, upload=39166]: Done uploading piece. err: <nil>
Aug 31 10:35:46.203261847 (44h27m21.556046995s) [noise=true , address=45.91.116.6:4973 , piece_id=VGG3ORU6E2PDMAGENBR2AJWRMQ22QLAFI5DAQUB4XMO4KI5G4TQQ, seg_pos={0 0}, upload=39166]: Done uploading piece. err: <nil>
Aug 31 10:36:15.930363208 (44h27m51.283148347s) [noise=true , address=77.90.31.17:4772 , piece_id=EXYIUPQYWHMWJX6YDTK7NMANOLDI6UPQK3SYUKPH4N42MED2CWYA, seg_pos={0 0}, upload=39166]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 31 10:36:15.930627486 (44h27m51.283412614s) [noise=true , address=77.90.31.16:4735 , piece_id=VT2UI46533CVPZLAMRC5TNZKQCCUWUWRBFXUUDZH7NWYVBYQXBJA, seg_pos={0 0}, upload=39166]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op
Aug 31 10:36:15.930934504 (44h27m51.283719632s) [noise=true , address=77.90.31.10:4517 , piece_id=ETZJZFDL3HNZA3GSJS4GE2QZZCLSVYBZEDD2CWZIARBQ7XBVKRHQ, seg_pos={0 0}, upload=39166]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 5.161.89.252:18584->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 31 10:36:15.931120965 (44h27m51.283906103s) [noise=true , address=77.90.31.14:4662 , piece_id=5GG7POTFX2VDBHTSDLLRARV5OQ5SNWUXASAFC734IGNLJIOR34EQ, seg_pos={0 0}, upload=39166]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 31 10:36:15.931311063 (44h27m51.284096192s) [noise=true , address=77.90.31.12:4590 , piece_id=ROSYARSOOVQHYE7VI4QQFXVTRWHXMZCVR2JS4WG2PTUBSXJDANGQ, seg_pos={0 0}, upload=39166]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:48956->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 31 10:36:15.931413836 (44h27m51.284198964s) [seg_pos={0 0}, upload=39166]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil>
Aug 31 10:36:15.954490543 (44h27m51.307275682s) [upload=39166]: Done uploading segments...
Aug 31 10:36:15.954518606 (44h27m51.307303744s) [upload=39166]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10)
Duration: 514.987147ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [..............XXX...........................................] duration:
356.082008ms
FAIL (2025-08-31 11:03:05 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 356.09931ms
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [X..............] duration:
76.77816ms remote:
"tcp:54.243.86.28:443" response headers:
Location=https://static.storj.io/dcs-satellites Cf-Ray=977bfa142fb0c920-IAD X-Cluster-Name=us-east-1-prod-hosting-red Date=Sun, 31 Aug 2025 10:52:10 GMT Content-Length=166 Cf-Cache-Status=BYPASS Cache-Control=private Vary=Accept-Encoding Set-Cookie=_cfuvid=X0HoV3xdSBD3_SuElPRxJP0DdcMIAj1ZolwS2hCRoUw-1756637530317-0.0.1.1-604800000; path=/; domain=.cdn.webflow.com; HttpOnly; Secure; SameSite=None Alt-Svc=h3=":443"; ma=86400 Content-Type=text/html target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-31 11:03:04 UTC)
orbiter warning: no valid certificate chain appears to exist after 336h0m0s
Duration: 76.800251ms
core
website
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)antiblip:
2 out of 60 is failed (threshold=3) [....................XX......................................] duration:
886.451031ms
FAIL (2025-08-31 11:03:02 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 886.469094ms
core
uplink
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
80.038622ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-08-31 11:03:01 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp: lookup uptime-checks.gateway.storjshare.io on 100.100.100.100:53: read udp 100.64.0.137:29236->100.100.100.100:53: i/o timeout
Duration: 80.109515ms
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
155.880057ms
FAIL (2025-08-31 11:02:59 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 155.906647ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....................XX.....................................] duration:
2.822881761s
FAIL (2025-08-31 11:02:55 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 2.822978614s
core
uplink
eu1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
4 out of 60 is failed (threshold=3) [.................XXXX.......................................] duration:
33.956601ms target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-31 11:02:55 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp 45.38.24.2:443: i/o timeout
Duration: 33.974915ms
edge
gateway
vhost
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
2 out of 60 is failed (threshold=3) [...................XX.......................................] duration:
181.559152ms
FAIL (2025-08-31 11:02:55 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 100.100.100.100:53: read udp 100.64.0.137:62187->100.100.100.100:53: i/o timeout
Duration: 181.578809ms
edge
authservice
http
bgp
us1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
4.816080841s
FAIL (2025-08-31 11:02:52 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 4.816106749s
core
uplink
ap1
|
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
696.709125ms target:
https://ap1.storj.io/login
FAIL (2025-08-31 11:02:51 UTC)
orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.157.232:443: i/o timeout
Duration: 696.728741ms
core
satellite
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
272.062101ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011
FAIL (2025-08-31 11:02:51 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": dial tcp: lookup uptime-checks.gateway.storjshare.io on 100.100.100.100:53: read udp 100.64.0.137:29236->100.100.100.100:53: i/o timeout
Duration: 272.082069ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)antiblip:
2 out of 15 is failed (threshold=6) [.............XX] duration:
61.016236ms remote:
"tcp:34.202.203.47:443" response headers:
Last-Modified=Tue, 26 Aug 2025 19:05:15 GMT Vary=Accept-Encoding Alt-Svc=h3=":443"; ma=86400 X-Lambda-Id=c6207948-cc73-40b2-9a2a-e948767f3d5e Set-Cookie=_cfuvid=jTuNIDpNmuoQL4fNtW4oS15EfCGUoGMeAeYlD9Sd9Cc-1756638165925-0.0.1.1-604800000; path=/; domain=.cdn.webflow.com; HttpOnly; Secure; SameSite=None Date=Sun, 31 Aug 2025 11:02:45 GMT Surrogate-Control=max-age=2147483647 Surrogate-Key=www.storj.io 65d78d3b8dbb1f54a94020c7 pageId:67eb0cf63d37c62ac04e0ff0 X-Cluster-Name=us-east-1-prod-hosting-red Content-Type=text/html Cf-Ray=977c0998b88e2070-IAD Cf-Cache-Status=HIT Age=403050 target:
https://www.storj.io/
FAIL (2025-08-31 11:02:45 UTC)
orbiter warning: no valid certificate chain appears to exist after 336h0m0s
Duration: 61.038918ms
core
website
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
133.920211ms
FAIL (2025-08-31 11:02:44 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 133.955188ms
core
uplink
us1
|