Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 21 out of 60 is failed (threshold=3) [...............................XXXXXXXXXXXXXXXX.X.XXXX......] duration: 428.228187ms FAIL (2025-08-07 22:32:40 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 428.253194ms core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 22 out of 60 is failed (threshold=3) [..X..............................XXXXXXXXXXXXXXXXX..XX..XX..] duration: 264.844279ms FAIL (2025-08-07 22:32:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 264.872421ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 19 out of 60 is failed (threshold=3) [XX.X.XX...............................XXXXXXXXXXX.X.XX......] duration: 309.8678ms FAIL (2025-08-07 22:32:35 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 04]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:19:04.377299713 (5h52m17.964355924s) [upload=6018]: Starting upload Aug 7 22:19:04.377350166 (5h52m17.964406387s) [upload=6018]: Done starting upload Aug 7 22:19:04.377401523 (5h52m17.964457744s) [upload=6018]: Uploading segments... Aug 7 22:19:04.377446897 (5h52m17.964503118s) [upload=6018]: Got next segment. Inline: true Aug 7 22:19:04.377453410 (5h52m17.96450962s) [upload=6018]: Waiting for error group managing segments... Aug 7 22:19:04.377487142 (5h52m17.964543354s) [upload=6018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:19:04.377491361 (5h52m17.964547582s) [upload=6018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:19:04.377493875 (5h52m17.964550086s) [upload=6018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:20:04.361964803 (5h53m17.949021024s) [upload=6018]: Done uploading segments... Aug 7 22:20:04.361979240 (5h53m17.949035461s) [upload=6018]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:20:11.847632631 (5h53m25.434688843s) [upload=6035]: Starting upload Aug 7 22:20:11.847694537 (5h53m25.434750747s) [upload=6035]: Done starting upload Aug 7 22:20:11.847740192 (5h53m25.434796403s) [upload=6035]: Uploading segments... Aug 7 22:20:11.847772231 (5h53m25.434828442s) [upload=6035]: Got next segment. Inline: true Aug 7 22:20:11.847776860 (5h53m25.434833071s) [upload=6035]: Waiting for error group managing segments... Aug 7 22:20:11.847824679 (5h53m25.434880891s) [upload=6035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:20:11.847830130 (5h53m25.434886341s) [upload=6035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:20:11.847833205 (5h53m25.434889417s) [upload=6035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:21:11.836000640 (5h54m25.42305688s) [upload=6035]: Done uploading segments... Aug 7 22:21:11.836026087 (5h54m25.423082328s) [upload=6035]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:21:11.965120248 (5h54m25.552176489s) [upload=6050]: Starting upload Aug 7 22:21:11.965231515 (5h54m25.552287757s) [upload=6050]: Done starting upload Aug 7 22:21:11.965301156 (5h54m25.552357397s) [upload=6050]: Uploading segments... Aug 7 22:21:11.965348053 (5h54m25.552404304s) [upload=6050]: Got next segment. Inline: true Aug 7 22:21:11.965353994 (5h54m25.552410235s) [upload=6050]: Waiting for error group managing segments... Aug 7 22:21:11.965399049 (5h54m25.55245529s) [upload=6050]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:21:11.965405170 (5h54m25.552461411s) [upload=6050]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:21:11.965408957 (5h54m25.552465198s) [upload=6050]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:22:11.952333050 (5h55m25.539389261s) [upload=6050]: Done uploading segments... Aug 7 22:22:11.952348108 (5h55m25.539404329s) [upload=6050]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:22:12.125652873 (5h55m25.712709094s) [upload=6063]: Starting upload Aug 7 22:22:12.125702476 (5h55m25.712758688s) [upload=6063]: Done starting upload Aug 7 22:22:12.125747020 (5h55m25.712803241s) [upload=6063]: Uploading segments... Aug 7 22:22:12.125776895 (5h55m25.712833116s) [upload=6063]: Got next segment. Inline: true Aug 7 22:22:12.125781404 (5h55m25.712837625s) [upload=6063]: Waiting for error group managing segments... Aug 7 22:22:12.125814315 (5h55m25.712870536s) [upload=6063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:22:12.125818052 (5h55m25.712874263s) [upload=6063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:22:12.125820397 (5h55m25.712876618s) [upload=6063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:23:12.024254277 (5h56m25.611310498s) [upload=6063]: Done uploading segments... Aug 7 22:23:12.024289072 (5h56m25.611345283s) [upload=6063]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:23:12.175471355 (5h56m25.762527616s) [upload=6082]: Starting upload Aug 7 22:23:12.175573175 (5h56m25.762629466s) [upload=6082]: Done starting upload Aug 7 22:23:12.175670356 (5h56m25.762726627s) [upload=6082]: Uploading segments... Aug 7 22:23:12.175760093 (5h56m25.762816364s) [upload=6082]: Got next segment. Inline: true Aug 7 22:23:12.175767677 (5h56m25.762823919s) [upload=6082]: Waiting for error group managing segments... Aug 7 22:23:12.175819345 (5h56m25.762875585s) [upload=6082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:23:12.175825105 (5h56m25.762881346s) [upload=6082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:23:12.175828732 (5h56m25.762884973s) [upload=6082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:24:12.154559214 (5h57m25.741615455s) [upload=6082]: Done uploading segments... Aug 7 22:24:12.154578851 (5h57m25.741635092s) [upload=6082]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:24:12.253523032 (5h57m25.840579253s) [upload=6098]: Starting upload Aug 7 22:24:12.253572454 (5h57m25.840628665s) [upload=6098]: Done starting upload Aug 7 22:24:12.253615034 (5h57m25.840671245s) [upload=6098]: Uploading segments... Aug 7 22:24:12.253654808 (5h57m25.840711029s) [upload=6098]: Got next segment. Inline: true Aug 7 22:24:12.253658636 (5h57m25.840714847s) [upload=6098]: Waiting for error group managing segments... Aug 7 22:24:12.253689122 (5h57m25.840745343s) [upload=6098]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:24:12.253692969 (5h57m25.84074918s) [upload=6098]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:12.253695303 (5h57m25.840751524s) [upload=6098]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:24:12.467343237 (5h57m26.054399448s) [upload=6098]: Done uploading segments... Aug 7 22:24:12.467365428 (5h57m26.05442164s) [upload=6098]: Upload finished. err: <nil> Aug 7 22:24:33.085790933 (5h57m46.672847184s) [upload=6106]: Starting upload Aug 7 22:24:33.085870281 (5h57m46.672926522s) [upload=6106]: Done starting upload Aug 7 22:24:33.085932668 (5h57m46.67298892s) [upload=6106]: Uploading segments... Aug 7 22:24:33.085994053 (5h57m46.673050293s) [upload=6106]: Got next segment. Inline: true Aug 7 22:24:33.086000064 (5h57m46.673056304s) [upload=6106]: Waiting for error group managing segments... Aug 7 22:24:33.086045709 (5h57m46.67310195s) [upload=6106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:24:33.086051168 (5h57m46.67310741s) [upload=6106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:33.086054985 (5h57m46.673111227s) [upload=6106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:33.072711385 (5h58m46.659767636s) [upload=6106]: Done uploading segments... Aug 7 22:25:33.072753384 (5h58m46.659809634s) [upload=6106]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:25:33.160012555 (5h58m46.747068795s) [upload=6120]: Starting upload Aug 7 22:25:33.160101981 (5h58m46.747158222s) [upload=6120]: Done starting upload Aug 7 22:25:33.160199473 (5h58m46.747255715s) [upload=6120]: Uploading segments... Aug 7 22:25:33.160243596 (5h58m46.747299837s) [upload=6120]: Got next segment. Inline: true Aug 7 22:25:33.160249046 (5h58m46.747305287s) [upload=6120]: Waiting for error group managing segments... Aug 7 22:25:33.160294231 (5h58m46.747350482s) [upload=6120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:25:33.160300012 (5h58m46.747356252s) [upload=6120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:25:33.160303619 (5h58m46.747359859s) [upload=6120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:33.228188865 (5h58m46.815245086s) [upload=6120]: Done uploading segments... Aug 7 22:25:33.228202571 (5h58m46.815258782s) [upload=6120]: Upload finished. err: <nil> Aug 7 22:26:14.972993755 (5h59m28.560049967s) [upload=6132]: Starting upload Aug 7 22:26:14.973047095 (5h59m28.560103307s) [upload=6132]: Done starting upload Aug 7 22:26:14.973101456 (5h59m28.560157677s) [upload=6132]: Uploading segments... Aug 7 22:26:14.973137704 (5h59m28.560193925s) [upload=6132]: Got next segment. Inline: true Aug 7 22:26:14.973141382 (5h59m28.560197592s) [upload=6132]: Waiting for error group managing segments... Aug 7 22:26:14.973187948 (5h59m28.560244158s) [upload=6132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:26:14.973191995 (5h59m28.560248217s) [upload=6132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:26:14.973195201 (5h59m28.560251423s) [upload=6132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:14.950324988 (6h0m28.537381229s) [upload=6132]: Done uploading segments... Aug 7 22:27:14.950345787 (6h0m28.537402028s) [upload=6132]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:27:22.925128081 (6h0m36.512184292s) [upload=6146]: Starting upload Aug 7 22:27:22.925194244 (6h0m36.512250465s) [upload=6146]: Done starting upload Aug 7 22:27:22.925235571 (6h0m36.512291783s) [upload=6146]: Uploading segments... Aug 7 22:27:22.925260508 (6h0m36.512316729s) [upload=6146]: Got next segment. Inline: true Aug 7 22:27:22.925264486 (6h0m36.512320707s) [upload=6146]: Waiting for error group managing segments... Aug 7 22:27:22.925305752 (6h0m36.512361973s) [upload=6146]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:22.925309750 (6h0m36.512365971s) [upload=6146]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:22.925312044 (6h0m36.512368265s) [upload=6146]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:28:22.913115749 (6h1m36.50017199s) [upload=6146]: Done uploading segments... Aug 7 22:28:22.913136929 (6h1m36.500193169s) [upload=6146]: Upload finished. err: metaclient: context deadline exceeded Duration: 309.89866ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 19 out of 60 is failed (threshold=3) [XX....................................XXXXXXXXXXXXXXXX..X...] duration: 1.864584888s FAIL (2025-08-07 22:32:35 UTC) operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.864602431s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [..................................XXXXXXXXXXXXXX.XXX........] duration: 485.183691ms FAIL (2025-08-07 22:32:33 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== TVRWLFHTOF2YZFDYX3LU5FENJAS6JOW572GLJ2OHVQ, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.749054955 (5h56m13.336111176s) [noise=true , address=104.36.82.2:4730 , piece_id=FKFZNBUC7L2WVTTFPNRSPJUWW6HAEE6CNUAYSHVDYXW5XENS462Q, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.749426248 (5h56m13.336482459s) [noise=true , address=45.91.116.9:4262 , piece_id=R7J2NVZJTXR7ATH2VUC75B2Z73TSJAQUBYJ3WE75RQ6AD3L3PSCQ, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.750681361 (5h56m13.337737582s) [noise=true , address=104.36.82.5:4840 , piece_id=6SYENOMYMIEYCEFSHJKMXOQE4RCDPDWBAHYDVRTXCQYWJGQDCCKA, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.751093369 (5h56m13.338149581s) [noise=true , address=104.36.82.14:4221 , piece_id=SBKEVRBOXXBFOVTZWPSDVDPXEDV6HZIVAV4MWFIYDMNEU643KBWA, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.751643326 (5h56m13.338699547s) [noise=true , address=45.91.116.4:4940 , piece_id=UXKWVJQC6KYUHGXKHRYHJATOP5KQF4BY2LWNQJQEGFYKQEBT6GWA, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.752249137 (5h56m13.339305348s) [noise=true , address=77.90.31.13:4628 , piece_id=BYQWTXNUD76XIWZZAUZNHDUX7F7CABMDCPK7M7QPDHS3CFBRBMVQ, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.752826465 (5h56m13.339882676s) [noise=true , address=104.36.82.2:4731 , piece_id=L6W56WDLHKGHEPXBK7F2OYZ6CRM66ZVKG5RGJPH3FHC7RZZ3QGBA, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.753473372 (5h56m13.340529583s) [noise=true , address=45.91.116.12:4227 , piece_id=7RIB2VCODFSHTRTBGI3Z74IECO3ATQXSXW7CY3TCA43MZKSKEQYA, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.753961884 (5h56m13.341018096s) [noise=true , address=77.90.31.10:4519 , piece_id=QF2SVPCWB5HG4S7GQL6NFHTYPVOHP4HHORRY2TMJSRXP5A3J43TQ, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.754616727 (5h56m13.341672948s) [noise=true , address=104.36.82.9:4006 , piece_id=MDXFW3M6TBQJXYHMBTPIOD5I5PK3NUHDRZ3EERNCRKFVMUPSW43Q, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.755122451 (5h56m13.342178672s) [noise=true , address=104.36.82.8:4971 , piece_id=2MZZBGEPYJSHOBAG4FUAU5NKKAOL72C2AWD3GZECQXGPMOG6TDMQ, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.837536198 (5h56m13.424592419s) [upload=6077]: Next returned nil segment Aug 7 22:22:59.837547418 (5h56m13.424603629s) [upload=6077]: Waiting for error group managing segments... Aug 7 22:22:59.846810959 (5h56m13.43386718s) [noise=true , address=45.91.116.7:4011 , piece_id=E7YGVXQXIO2A34B22AHGKBQXER4KLEX3AFGHVFUQ6USLQMUFSW2Q, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.847279324 (5h56m13.434335535s) [noise=true , address=45.91.116.12:4227 , piece_id=7RIB2VCODFSHTRTBGI3Z74IECO3ATQXSXW7CY3TCA43MZKSKEQYA, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.848559454 (5h56m13.435615675s) [noise=true , address=45.91.116.4:4940 , piece_id=UXKWVJQC6KYUHGXKHRYHJATOP5KQF4BY2LWNQJQEGFYKQEBT6GWA, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.852382881 (5h56m13.439439102s) [noise=true , address=45.91.116.9:4262 , piece_id=R7J2NVZJTXR7ATH2VUC75B2Z73TSJAQUBYJ3WE75RQ6AD3L3PSCQ, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.852577974 (5h56m13.439634195s) [noise=true , address=77.90.31.19:4842 , piece_id=ULCHXICRO3ULX7IEW6GVXHHGXT6F42N3WQDEEUVOSKCXID6F6XZQ, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.856642091 (5h56m13.443698302s) [noise=true , address=104.36.82.8:4971 , piece_id=2MZZBGEPYJSHOBAG4FUAU5NKKAOL72C2AWD3GZECQXGPMOG6TDMQ, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.856766534 (5h56m13.443822745s) [noise=true , address=77.90.31.10:4519 , piece_id=QF2SVPCWB5HG4S7GQL6NFHTYPVOHP4HHORRY2TMJSRXP5A3J43TQ, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.868085922 (5h56m13.455142143s) [noise=true , address=45.91.116.12:4225 , piece_id=MNJGBICBIG3HQXCO7B2C2ITRFSOW475RAGNVTAQNVFQYAJ7DOU5Q, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.868246762 (5h56m13.455302973s) [noise=true , address=104.36.82.2:4730 , piece_id=FKFZNBUC7L2WVTTFPNRSPJUWW6HAEE6CNUAYSHVDYXW5XENS462Q, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.877804453 (5h56m13.464860674s) [noise=true , address=104.36.82.11:4115 , piece_id=BALIRX6S5WW2CGSXSIFHWV2ABDH3ASPHR36YEYRUYTTGZCW6U6AQ, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.882357252 (5h56m13.469413462s) [noise=true , address=104.36.82.2:4731 , piece_id=L6W56WDLHKGHEPXBK7F2OYZ6CRM66ZVKG5RGJPH3FHC7RZZ3QGBA, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.887398722 (5h56m13.474454933s) [noise=true , address=104.36.82.14:4221 , piece_id=SBKEVRBOXXBFOVTZWPSDVDPXEDV6HZIVAV4MWFIYDMNEU643KBWA, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.887674137 (5h56m13.474730358s) [noise=true , address=104.36.82.10:4079 , piece_id=DHLJEMGMQUTVRWLFHTOF2YZFDYX3LU5FENJAS6JOW572GLJ2OHVQ, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.887689665 (5h56m13.474745877s) [seg_pos={0 0}, upload=6077]: Segment reached optimal threshold of 13 pieces. Aug 7 22:22:59.887775385 (5h56m13.474831606s) [noise=true , address=77.90.31.13:4628 , piece_id=BYQWTXNUD76XIWZZAUZNHDUX7F7CABMDCPK7M7QPDHS3CFBRBMVQ, seg_pos={0 0}, upload=6077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: expected piece hash; context canceled; EOF Aug 7 22:22:59.887795453 (5h56m13.474851674s) [noise=true , address=104.36.82.5:4840 , piece_id=6SYENOMYMIEYCEFSHJKMXOQE4RCDPDWBAHYDVRTXCQYWJGQDCCKA, seg_pos={0 0}, upload=6077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Aug 7 22:22:59.887879469 (5h56m13.47493569s) [noise=true , address=104.36.82.9:4006 , piece_id=MDXFW3M6TBQJXYHMBTPIOD5I5PK3NUHDRZ3EERNCRKFVMUPSW43Q, seg_pos={0 0}, upload=6077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc): protocol: write tcp 5.161.89.252:59712->104.36.82.9:4006: use of closed network connection; write tcp 5.161.89.252:59712->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:59712->104.36.82.9:4006: use of closed network connection Aug 7 22:22:59.887943669 (5h56m13.47499988s) [seg_pos={0 0}, upload=6077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 22:22:59.888006888 (5h56m13.475063109s) [upload=6077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:22:59.888010665 (5h56m13.475066886s) [upload=6077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:22:59.933305033 (5h56m13.520361243s) [upload=6077]: Done uploading segments... Aug 7 22:22:59.933317746 (5h56m13.520373958s) [upload=6077]: Upload finished. err: <nil> Aug 7 22:23:47.739190890 (5h57m1.326247131s) [upload=6091]: Starting upload Aug 7 22:23:47.739297178 (5h57m1.326353419s) [upload=6091]: Done starting upload Aug 7 22:23:47.739428053 (5h57m1.326484274s) [upload=6091]: Uploading segments... Aug 7 22:23:47.739777785 (5h57m1.326834025s) [upload=6091]: Got next segment. Inline: false Aug 7 22:23:47.739805196 (5h57m1.326861407s) [upload=6091]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:23:47.739811638 (5h57m1.326867849s) [upload=6091]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:24:17.722308743 (5h57m31.309364985s) [upload=6091]: Next segment err: context deadline exceeded Aug 7 22:24:17.722438996 (5h57m31.309495238s) [upload=6091]: Done uploading segments... Aug 7 22:24:17.722476005 (5h57m31.309532246s) [upload=6091]: Upload finished. err: context deadline exceeded Aug 7 22:24:18.676382471 (5h57m32.263438711s) [upload=6101]: Starting upload Aug 7 22:24:18.676482739 (5h57m32.263538979s) [upload=6101]: Done starting upload Aug 7 22:24:18.676619723 (5h57m32.263675964s) [upload=6101]: Uploading segments... Aug 7 22:24:18.676886642 (5h57m32.263942883s) [upload=6101]: Got next segment. Inline: false Aug 7 22:24:18.676922940 (5h57m32.263979181s) [upload=6101]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:24:18.676930835 (5h57m32.263987076s) [upload=6101]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:24:48.662510024 (5h58m2.249566255s) [upload=6101]: Next segment err: context deadline exceeded Aug 7 22:24:48.662708235 (5h58m2.249764465s) [upload=6101]: Done uploading segments... Aug 7 22:24:48.662735284 (5h58m2.249791526s) [upload=6101]: Upload finished. err: context deadline exceeded Aug 7 22:25:18.142081018 (5h58m31.729137259s) [upload=6117]: Starting upload Aug 7 22:25:18.142160586 (5h58m31.729216828s) [upload=6117]: Done starting upload Aug 7 22:25:18.142286381 (5h58m31.729342623s) [upload=6117]: Uploading segments... Aug 7 22:25:18.142594587 (5h58m31.729650828s) [upload=6117]: Got next segment. Inline: false Aug 7 22:25:18.142641995 (5h58m31.729698236s) [upload=6117]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:25:18.142647436 (5h58m31.729703676s) [upload=6117]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:25:48.125202589 (5h59m1.71225884s) [upload=6117]: Next segment err: context deadline exceeded Aug 7 22:25:48.125308446 (5h59m1.712364688s) [upload=6117]: Done uploading segments... Aug 7 22:25:48.125355273 (5h59m1.712411525s) [upload=6117]: Upload finished. err: context deadline exceeded Duration: 485.198048ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 396.31267ms FAIL (2025-08-07 22:32:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ine Aug 7 21:59:56.515336538 (5h33m10.10239275s) [upload=5708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:59:56.574152557 (5h33m10.161208778s) [upload=5708]: Done uploading segments... Aug 7 21:59:56.574171943 (5h33m10.161228164s) [upload=5708]: Upload finished. err: <nil> Aug 7 22:00:36.611785100 (5h33m50.19884134s) [upload=5716]: Starting upload Aug 7 22:00:36.611856203 (5h33m50.198912444s) [upload=5716]: Done starting upload Aug 7 22:00:36.611959786 (5h33m50.199016027s) [upload=5716]: Uploading segments... Aug 7 22:00:36.612003668 (5h33m50.199059918s) [upload=5716]: Got next segment. Inline: true Aug 7 22:00:36.612009668 (5h33m50.19906591s) [upload=5716]: Waiting for error group managing segments... Aug 7 22:00:36.612057077 (5h33m50.199113319s) [upload=5716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:00:36.612082536 (5h33m50.199138776s) [upload=5716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:00:36.612086222 (5h33m50.199142473s) [upload=5716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:00:36.884781084 (5h33m50.471837294s) [upload=5716]: Done uploading segments... Aug 7 22:00:36.884795611 (5h33m50.471851832s) [upload=5716]: Upload finished. err: <nil> Aug 7 22:01:47.555449092 (5h35m1.142505313s) [upload=5736]: Starting upload Aug 7 22:01:47.555491781 (5h35m1.142548002s) [upload=5736]: Done starting upload Aug 7 22:01:47.555562123 (5h35m1.142618344s) [upload=5736]: Uploading segments... Aug 7 22:01:47.555595826 (5h35m1.142652047s) [upload=5736]: Got next segment. Inline: true Aug 7 22:01:47.555599363 (5h35m1.142655584s) [upload=5736]: Waiting for error group managing segments... Aug 7 22:01:47.555631602 (5h35m1.142687823s) [upload=5736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:47.555635390 (5h35m1.142691611s) [upload=5736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:47.555637894 (5h35m1.142694115s) [upload=5736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:47.686110188 (5h35m1.273166429s) [upload=5736]: Done uploading segments... Aug 7 22:01:47.686151165 (5h35m1.273207406s) [upload=5736]: Upload finished. err: <nil> Aug 7 22:02:22.678566055 (5h35m36.265622267s) [upload=5746]: Starting upload Aug 7 22:02:22.678633091 (5h35m36.265689303s) [upload=5746]: Done starting upload Aug 7 22:02:22.678684407 (5h35m36.265740617s) [upload=5746]: Uploading segments... Aug 7 22:02:22.678721757 (5h35m36.265777967s) [upload=5746]: Got next segment. Inline: true Aug 7 22:02:22.678725724 (5h35m36.265781935s) [upload=5746]: Waiting for error group managing segments... Aug 7 22:02:22.678759457 (5h35m36.265815668s) [upload=5746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:02:22.678764947 (5h35m36.265821168s) [upload=5746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:02:22.678768565 (5h35m36.265824826s) [upload=5746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:02:22.971349697 (5h35m36.558405918s) [upload=5746]: Done uploading segments... Aug 7 22:02:22.971373742 (5h35m36.558429963s) [upload=5746]: Upload finished. err: <nil> Aug 7 22:03:08.907032448 (5h36m22.494088689s) [upload=5757]: Starting upload Aug 7 22:03:08.907091528 (5h36m22.494147769s) [upload=5757]: Done starting upload Aug 7 22:03:08.907172368 (5h36m22.4942286s) [upload=5757]: Uploading segments... Aug 7 22:03:08.907209458 (5h36m22.494265699s) [upload=5757]: Got next segment. Inline: true Aug 7 22:03:08.907214758 (5h36m22.494271s) [upload=5757]: Waiting for error group managing segments... Aug 7 22:03:08.907255936 (5h36m22.494312176s) [upload=5757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:03:08.907261175 (5h36m22.494317416s) [upload=5757]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:03:08.907265904 (5h36m22.494322135s) [upload=5757]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:03:09.004207779 (5h36m22.59126404s) [upload=5757]: Done uploading segments... Aug 7 22:03:09.004232375 (5h36m22.591288627s) [upload=5757]: Upload finished. err: <nil> Aug 7 22:04:02.463086309 (5h37m16.050142521s) [upload=5775]: Starting upload Aug 7 22:04:02.463128347 (5h37m16.050184568s) [upload=5775]: Done starting upload Aug 7 22:04:02.463197818 (5h37m16.050254029s) [upload=5775]: Uploading segments... Aug 7 22:04:02.463223595 (5h37m16.050279816s) [upload=5775]: Got next segment. Inline: true Aug 7 22:04:02.463227652 (5h37m16.050283873s) [upload=5775]: Waiting for error group managing segments... Aug 7 22:04:02.463258120 (5h37m16.050314331s) [upload=5775]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:04:02.463261796 (5h37m16.050318008s) [upload=5775]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:04:02.463264733 (5h37m16.050320954s) [upload=5775]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:04:02.530547234 (5h37m16.117603445s) [upload=5775]: Done uploading segments... Aug 7 22:04:02.530562512 (5h37m16.117618733s) [upload=5775]: Upload finished. err: <nil> Aug 7 22:05:10.744065032 (5h38m24.331121272s) [upload=5796]: Starting upload Aug 7 22:05:10.744195716 (5h38m24.331251957s) [upload=5796]: Done starting upload Aug 7 22:05:10.744327762 (5h38m24.331384002s) [upload=5796]: Uploading segments... Aug 7 22:05:10.744369549 (5h38m24.331425791s) [upload=5796]: Got next segment. Inline: true Aug 7 22:05:10.744388655 (5h38m24.331444897s) [upload=5796]: Waiting for error group managing segments... Aug 7 22:05:10.744449028 (5h38m24.331505259s) [upload=5796]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:05:10.744454398 (5h38m24.331510629s) [upload=5796]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:05:10.744457633 (5h38m24.331513875s) [upload=5796]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:05:10.852991578 (5h38m24.440047789s) [upload=5796]: Done uploading segments... Aug 7 22:05:10.853004571 (5h38m24.440060782s) [upload=5796]: Upload finished. err: <nil> Aug 7 22:05:45.060720865 (5h38m58.647777086s) [upload=5806]: Starting upload Aug 7 22:05:45.060780046 (5h38m58.647836267s) [upload=5806]: Done starting upload Aug 7 22:05:45.060842121 (5h38m58.647898333s) [upload=5806]: Uploading segments... Aug 7 22:05:45.060876836 (5h38m58.647933057s) [upload=5806]: Got next segment. Inline: true Aug 7 22:05:45.060882878 (5h38m58.647939099s) [upload=5806]: Waiting for error group managing segments... Aug 7 22:05:45.060927901 (5h38m58.647984112s) [upload=5806]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:05:45.060933843 (5h38m58.647990064s) [upload=5806]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:05:45.060936187 (5h38m58.647992408s) [upload=5806]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:05:45.328399453 (5h38m58.915455674s) [upload=5806]: Done uploading segments... Aug 7 22:05:45.328433367 (5h38m58.915489588s) [upload=5806]: Upload finished. err: <nil> Aug 7 22:06:12.931860875 (5h39m26.518917096s) [upload=5815]: Starting upload Aug 7 22:06:12.931905979 (5h39m26.51896219s) [upload=5815]: Done starting upload Aug 7 22:06:12.931964518 (5h39m26.51902073s) [upload=5815]: Uploading segments... Aug 7 22:06:12.931990617 (5h39m26.519046827s) [upload=5815]: Got next segment. Inline: true Aug 7 22:06:12.931994224 (5h39m26.519050434s) [upload=5815]: Waiting for error group managing segments... Aug 7 22:06:12.932024821 (5h39m26.519081042s) [upload=5815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:06:12.932028408 (5h39m26.519084618s) [upload=5815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:06:12.932030893 (5h39m26.519087103s) [upload=5815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:06:13.014995076 (5h39m26.602051317s) [upload=5815]: Done uploading segments... Aug 7 22:06:13.015014863 (5h39m26.602071114s) [upload=5815]: Upload finished. err: <nil> Aug 7 22:07:16.916734820 (5h40m30.503791062s) [upload=5832]: Starting upload Aug 7 22:07:16.916800323 (5h40m30.503856563s) [upload=5832]: Done starting upload Aug 7 22:07:16.916864883 (5h40m30.503921123s) [upload=5832]: Uploading segments... Aug 7 22:07:16.916903304 (5h40m30.503959546s) [upload=5832]: Got next segment. Inline: true Aug 7 22:07:16.916908815 (5h40m30.503965056s) [upload=5832]: Waiting for error group managing segments... Aug 7 22:07:16.916952356 (5h40m30.504008598s) [upload=5832]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:07:16.916957657 (5h40m30.504013897s) [upload=5832]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:07:16.916961754 (5h40m30.504017995s) [upload=5832]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:07:16.988776316 (5h40m30.575832566s) [upload=5832]: Done uploading segments... Aug 7 22:07:16.988801573 (5h40m30.575857824s) [upload=5832]: Upload finished. err: <nil> Aug 7 22:08:29.650294124 (5h41m43.237350365s) [upload=5854]: Starting upload Aug 7 22:08:29.650357383 (5h41m43.237413623s) [upload=5854]: Done starting upload Aug 7 22:08:29.650418325 (5h41m43.237474567s) [upload=5854]: Uploading segments... Aug 7 22:08:29.650468490 (5h41m43.237524731s) [upload=5854]: Got next segment. Inline: true Aug 7 22:08:29.650474350 (5h41m43.237530592s) [upload=5854]: Waiting for error group managing segments... Aug 7 22:08:29.650515618 (5h41m43.237571859s) [upload=5854]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:08:29.650522029 (5h41m43.237578271s) [upload=5854]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:08:29.650525556 (5h41m43.237581797s) [upload=5854]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:08:29.706715553 (5h41m43.293771793s) [upload=5854]: Done uploading segments... Aug 7 22:08:29.706731001 (5h41m43.293787243s) [upload=5854]: Upload finished. err: metaclient: Too Many Requests Duration:* 396.323301ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 20 out of 60 is failed (threshold=3) [XXXXXXXXXXX..................................XXXXXXXX.X.....] duration: 1.689667941s FAIL (2025-08-07 22:32:32 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.689700511s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 25 out of 60 is failed (threshold=3) [XXX.............................XXXXXXXXXXXXXXXX.XXXXX.X....] duration: 197.192415ms FAIL (2025-08-07 22:32:27 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 420501s) [upload=6055]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:22:06.462569787 (5h55m20.049626008s) [upload=6061]: Starting upload Aug 7 22:22:06.462655577 (5h55m20.049711788s) [upload=6061]: Done starting upload Aug 7 22:22:06.462750936 (5h55m20.049807157s) [upload=6061]: Uploading segments... Aug 7 22:22:06.462782464 (5h55m20.049838675s) [upload=6061]: Got next segment. Inline: true Aug 7 22:22:06.462786181 (5h55m20.049842402s) [upload=6061]: Waiting for error group managing segments... Aug 7 22:22:06.462817791 (5h55m20.049874001s) [upload=6061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:22:06.462821648 (5h55m20.049877869s) [upload=6061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:22:06.462823982 (5h55m20.049880203s) [upload=6061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:22:36.449657257 (5h55m50.036713498s) [upload=6061]: Done uploading segments... Aug 7 22:22:36.449678196 (5h55m50.036734448s) [upload=6061]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:22:59.911739779 (5h56m13.498795989s) [upload=6078]: Starting upload Aug 7 22:22:59.911786736 (5h56m13.498842957s) [upload=6078]: Done starting upload Aug 7 22:22:59.911827472 (5h56m13.498883683s) [upload=6078]: Uploading segments... Aug 7 22:22:59.911861074 (5h56m13.498917295s) [upload=6078]: Got next segment. Inline: true Aug 7 22:22:59.911864401 (5h56m13.498920622s) [upload=6078]: Waiting for error group managing segments... Aug 7 22:22:59.911889107 (5h56m13.498945319s) [upload=6078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:22:59.911893005 (5h56m13.498949226s) [upload=6078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:22:59.911894898 (5h56m13.498951119s) [upload=6078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:23:29.900458747 (5h56m43.487514958s) [upload=6078]: Done uploading segments... Aug 7 22:23:29.900473835 (5h56m43.487530047s) [upload=6078]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:23:58.370463254 (5h57m11.957519465s) [upload=6094]: Starting upload Aug 7 22:23:58.370512907 (5h57m11.957569128s) [upload=6094]: Done starting upload Aug 7 22:23:58.370553031 (5h57m11.957609243s) [upload=6094]: Uploading segments... Aug 7 22:23:58.370578639 (5h57m11.95763486s) [upload=6094]: Got next segment. Inline: true Aug 7 22:23:58.370582427 (5h57m11.957638648s) [upload=6094]: Waiting for error group managing segments... Aug 7 22:23:58.370610951 (5h57m11.957667172s) [upload=6094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:23:58.370615178 (5h57m11.957671399s) [upload=6094]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:23:58.370617392 (5h57m11.957673613s) [upload=6094]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:23:58.450972793 (5h57m12.038029004s) [upload=6094]: Done uploading segments... Aug 7 22:23:58.450986078 (5h57m12.038042289s) [upload=6094]: Upload finished. err: <nil> Aug 7 22:24:40.794473113 (5h57m54.381529325s) [upload=6107]: Starting upload Aug 7 22:24:40.794523428 (5h57m54.381579639s) [upload=6107]: Done starting upload Aug 7 22:24:40.794565206 (5h57m54.381621417s) [upload=6107]: Uploading segments... Aug 7 22:24:40.794591815 (5h57m54.381648026s) [upload=6107]: Got next segment. Inline: true Aug 7 22:24:40.794595622 (5h57m54.381651833s) [upload=6107]: Waiting for error group managing segments... Aug 7 22:24:40.794624367 (5h57m54.381680577s) [upload=6107]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:24:40.794628323 (5h57m54.381684535s) [upload=6107]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:40.794630638 (5h57m54.381686849s) [upload=6107]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:10.774549311 (5h58m24.361605533s) [upload=6107]: Done uploading segments... Aug 7 22:25:10.774570019 (5h58m24.361626231s) [upload=6107]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:25:10.855666099 (5h58m24.442722339s) [upload=6114]: Starting upload Aug 7 22:25:10.855752680 (5h58m24.442808921s) [upload=6114]: Done starting upload Aug 7 22:25:10.855822740 (5h58m24.442878991s) [upload=6114]: Uploading segments... Aug 7 22:25:10.855866683 (5h58m24.442922923s) [upload=6114]: Got next segment. Inline: true Aug 7 22:25:10.855872564 (5h58m24.442928805s) [upload=6114]: Waiting for error group managing segments... Aug 7 22:25:10.855966008 (5h58m24.443022249s) [upload=6114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:25:10.855972971 (5h58m24.443029212s) [upload=6114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:25:10.855976568 (5h58m24.443032819s) [upload=6114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:40.842457902 (5h58m54.429514153s) [upload=6114]: Done uploading segments... Aug 7 22:25:40.842479021 (5h58m54.429535272s) [upload=6114]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:25:40.940807525 (5h58m54.527863766s) [upload=6122]: Starting upload Aug 7 22:25:40.940896992 (5h58m54.527953233s) [upload=6122]: Done starting upload Aug 7 22:25:40.940964388 (5h58m54.528020628s) [upload=6122]: Uploading segments... Aug 7 22:25:40.941010434 (5h58m54.528066675s) [upload=6122]: Got next segment. Inline: true Aug 7 22:25:40.941016375 (5h58m54.528072616s) [upload=6122]: Waiting for error group managing segments... Aug 7 22:25:40.941060899 (5h58m54.52811715s) [upload=6122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:25:40.941068091 (5h58m54.528124332s) [upload=6122]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:25:40.941073060 (5h58m54.528129311s) [upload=6122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:26:10.929528068 (5h59m24.516584319s) [upload=6122]: Done uploading segments... Aug 7 22:26:10.929548557 (5h59m24.516604797s) [upload=6122]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:26:11.008231659 (5h59m24.59528788s) [upload=6129]: Starting upload Aug 7 22:26:11.008286501 (5h59m24.595342713s) [upload=6129]: Done starting upload Aug 7 22:26:11.008333198 (5h59m24.595389419s) [upload=6129]: Uploading segments... Aug 7 22:26:11.008364838 (5h59m24.595421059s) [upload=6129]: Got next segment. Inline: true Aug 7 22:26:11.008368595 (5h59m24.595424816s) [upload=6129]: Waiting for error group managing segments... Aug 7 22:26:11.008399943 (5h59m24.595456164s) [upload=6129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:26:11.008403599 (5h59m24.59545981s) [upload=6129]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:26:11.008406565 (5h59m24.595462786s) [upload=6129]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:26:40.996305184 (5h59m54.583361385s) [upload=6129]: Done uploading segments... Aug 7 22:26:40.996323769 (5h59m54.58337998s) [upload=6129]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:27:14.829073707 (6h0m28.416129948s) [upload=6144]: Starting upload Aug 7 22:27:14.829153145 (6h0m28.416209386s) [upload=6144]: Done starting upload Aug 7 22:27:14.829247050 (6h0m28.416303291s) [upload=6144]: Uploading segments... Aug 7 22:27:14.829299899 (6h0m28.41635614s) [upload=6144]: Got next segment. Inline: true Aug 7 22:27:14.829306702 (6h0m28.416362943s) [upload=6144]: Waiting for error group managing segments... Aug 7 22:27:14.829350864 (6h0m28.416407105s) [upload=6144]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:14.829357416 (6h0m28.416413657s) [upload=6144]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:14.829361143 (6h0m28.416417384s) [upload=6144]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:44.808549878 (6h0m58.395606099s) [upload=6144]: Done uploading segments... Aug 7 22:27:44.808564877 (6h0m58.395621098s) [upload=6144]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:28:23.619901331 (6h1m37.206957573s) [upload=6162]: Starting upload Aug 7 22:28:23.620017008 (6h1m37.207073248s) [upload=6162]: Done starting upload Aug 7 22:28:23.620072551 (6h1m37.207128791s) [upload=6162]: Uploading segments... Aug 7 22:28:23.620125059 (6h1m37.2071813s) [upload=6162]: Got next segment. Inline: true Aug 7 22:28:23.620132432 (6h1m37.207188672s) [upload=6162]: Waiting for error group managing segments... Aug 7 22:28:23.620198335 (6h1m37.207254577s) [upload=6162]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:28:23.620204407 (6h1m37.207260647s) [upload=6162]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:28:23.620208153 (6h1m37.207264395s) [upload=6162]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:28:23.683041871 (6h1m37.270098112s) [upload=6162]: Done uploading segments... Aug 7 22:28:23.683055537 (6h1m37.270111778s) [upload=6162]: Upload finished. err: <nil> Aug 7 22:28:40.804037302 (6h1m54.391093523s) [upload=6165]: Starting upload Aug 7 22:28:40.804089460 (6h1m54.391145671s) [upload=6165]: Done starting upload Aug 7 22:28:40.804134294 (6h1m54.391190515s) [upload=6165]: Uploading segments... Aug 7 22:28:40.804181422 (6h1m54.391237632s) [upload=6165]: Got next segment. Inline: true Aug 7 22:28:40.804185379 (6h1m54.3912416s) [upload=6165]: Waiting for error group managing segments... Aug 7 22:28:40.804217198 (6h1m54.391273419s) [upload=6165]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:28:40.804220915 (6h1m54.391277127s) [upload=6165]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:28:40.804223370 (6h1m54.391279591s) [upload=6165]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:29:10.793781791 (6h2m24.380838023s) [upload=6165]: Done uploading segments... Aug 7 22:29:10.793803392 (6h2m24.380859603s) [upload=6165]: Upload finished. err: metaclient: context deadline exceeded Duration: 197.213535ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 24 out of 60 is failed (threshold=3) [................................XXXXXXXXXXXXXXXXXXX..XXXX.X.] duration: 379.827759ms FAIL (2025-08-07 22:32:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 379.846253ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 19 out of 60 is failed (threshold=3) [XXX.XX......X.............................XXXXXXXXXXXXX.....] duration: 1.72273053s FAIL (2025-08-07 22:32:21 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== [upload=6089]: Done starting upload Aug 7 22:23:43.961837605 (5h56m57.548893826s) [upload=6089]: Uploading segments... Aug 7 22:23:43.962138106 (5h56m57.549194347s) [upload=6089]: Got next segment. Inline: false Aug 7 22:23:43.962204581 (5h56m57.549260822s) [upload=6089]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:23:43.962209980 (5h56m57.549266222s) [upload=6089]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:24:12.375269699 (5h57m25.96232592s) [seg_pos={0 0}, upload=6089]: Begin upload segment... Aug 7 22:24:12.375313250 (5h57m25.962369471s) [seg_pos={0 0}, upload=6089]: Done begin upload segment. Aug 7 22:24:12.375339780 (5h57m25.962396031s) [noise=true , address=193.149.164.143:4698 , piece_id=AV2DCKYWWT5N4PL57WTXW45JNFYOCRLLEY6YSBSWOAQAPHAVXJQQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.375356050 (5h57m25.962412261s) [noise=true , address=104.36.82.53:4156 , piece_id=NBWZ4FB4S7CTDN2OTWTDKSH2R7DJN66P7GRTQTQ5FJZFQCZLDAQQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.375715481 (5h57m25.962771693s) [noise=true , address=104.36.82.58:4280 , piece_id=W2A65H4FIUOCUSMSVCANR2OPVBUFO4YNAFZV3KCASSCSPCHONGBA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.375724769 (5h57m25.96278099s) [noise=true , address=104.36.82.59:4333 , piece_id=JONL6XDWEUYMHVDPJVB54PE5HCYWN7KRO3FSNAMPJWCEP6QBKYVQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376022925 (5h57m25.963079146s) [noise=true , address=45.91.116.9:4270 , piece_id=CMJ7K7XDV44UUOWXU6NOUNMWZGLOEZKPTLVFMTNWIR6OK52TOEKQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376075263 (5h57m25.963131475s) [noise=true , address=104.36.82.36:4652 , piece_id=ZDJ2LSAX25MBAAW4PW745SX7J7X3S4PAGXEUS36T43KTM3LZ2WRQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376490167 (5h57m25.963546388s) [noise=true , address=104.36.82.45:4921 , piece_id=DCEBY7YTNAO4UR6YZ5LC5W27INFXZQ26BPJ7BHPMXY26P5BDL23Q, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376527327 (5h57m25.963583599s) [noise=true , address=104.36.82.12:4169 , piece_id=XPNXIWBPTKCEEQTVQ3BAQ3HCLQE5U5BTRIAR3FQ6O4OQJZMCJ6ZA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376755913 (5h57m25.963812125s) [noise=true , address=104.36.82.62:4411 , piece_id=3TMITFZMXF5ESOCJNLFU6EIT7CFUYULYZ6FJUQ6X7T2WBEQMKWFQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376864006 (5h57m25.963920216s) [noise=true , address=104.36.82.14:4243 , piece_id=WKT5D56ZXVSJJHVR3GB6N5XR6CGU4VW2XK2YTMPBZOH4S2ZPOVCA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376979641 (5h57m25.964035862s) [noise=true , address=104.36.82.9:4031 , piece_id=UOVVYSA77DQOK53ATDDXV56PLEVS6GBQ5HMZEKCEURYBPRCXNYPA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.377222374 (5h57m25.964278595s) [noise=true , address=104.36.82.52:4100 , piece_id=7RUPILCVG3GL3KB4CS25YCYKDB4C6HFT6TGZWZIDC5EHBI2SD55Q, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.377246499 (5h57m25.96430271s) [noise=true , address=79.127.201.215:4343 , piece_id=N3KGRRGXWJLCJGYSLPOLOIYZHP7AFWV6VBCHVBETDYOQBIXIWPMQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.377620267 (5h57m25.964676488s) [noise=true , address=104.36.82.35:4597 , piece_id=6UMSPJQOVLEDWMIQMLAJLZ2AY6Y5XLMODCN3V63EB5JNJS3EC7BA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.377642138 (5h57m25.964698349s) [noise=true , address=104.36.82.16:4300 , piece_id=RJRU7SETYXWMQK7GEE3HBD2XMEVSI3UJ6ZDCIWVW4266QSNAOPZA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.377932059 (5h57m25.96498828s) [noise=true , address=104.36.82.47:4544 , piece_id=DVQRASYHRKPFEONZEWJ3MBVM7O5764NPWYADLMG4ARAH5PBD55MA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.377950764 (5h57m25.965006985s) [noise=true , address=104.36.82.21:4478 , piece_id=5UO756WUOKCS52UJI2NIQFUSU7W6NBW34MGFH6YZX27HKWY5243A, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.378280479 (5h57m25.9653367s) [noise=true , address=104.36.82.37:4656 , piece_id=A25QOY4UFT7RRRY6PVAS3NNML7PGJM4OHDH56PF2HZHKJ7VOPURA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.468526485 (5h57m26.055582696s) [upload=6089]: Next returned nil segment Aug 7 22:24:12.468535081 (5h57m26.055591292s) [upload=6089]: Waiting for error group managing segments... Aug 7 22:24:12.483383738 (5h57m26.070439949s) [noise=true , address=104.36.82.37:4656 , piece_id=A25QOY4UFT7RRRY6PVAS3NNML7PGJM4OHDH56PF2HZHKJ7VOPURA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.500821799 (5h57m26.08787801s) [noise=true , address=104.36.82.16:4300 , piece_id=RJRU7SETYXWMQK7GEE3HBD2XMEVSI3UJ6ZDCIWVW4266QSNAOPZA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.504410257 (5h57m26.091466458s) [noise=true , address=45.91.116.9:4270 , piece_id=CMJ7K7XDV44UUOWXU6NOUNMWZGLOEZKPTLVFMTNWIR6OK52TOEKQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.511708921 (5h57m26.098765132s) [noise=true , address=104.36.82.35:4597 , piece_id=6UMSPJQOVLEDWMIQMLAJLZ2AY6Y5XLMODCN3V63EB5JNJS3EC7BA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.523758022 (5h57m26.110814233s) [noise=true , address=104.36.82.12:4169 , piece_id=XPNXIWBPTKCEEQTVQ3BAQ3HCLQE5U5BTRIAR3FQ6O4OQJZMCJ6ZA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.533614250 (5h57m26.120670461s) [noise=true , address=104.36.82.52:4100 , piece_id=7RUPILCVG3GL3KB4CS25YCYKDB4C6HFT6TGZWZIDC5EHBI2SD55Q, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.557032764 (5h57m26.144088985s) [noise=true , address=104.36.82.21:4478 , piece_id=5UO756WUOKCS52UJI2NIQFUSU7W6NBW34MGFH6YZX27HKWY5243A, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.582698011 (5h57m26.169754223s) [noise=true , address=104.36.82.53:4156 , piece_id=NBWZ4FB4S7CTDN2OTWTDKSH2R7DJN66P7GRTQTQ5FJZFQCZLDAQQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.609343038 (5h57m26.196399279s) [noise=true , address=79.127.201.215:4343 , piece_id=N3KGRRGXWJLCJGYSLPOLOIYZHP7AFWV6VBCHVBETDYOQBIXIWPMQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.639134328 (5h57m26.226190538s) [noise=true , address=104.36.82.36:4652 , piece_id=ZDJ2LSAX25MBAAW4PW745SX7J7X3S4PAGXEUS36T43KTM3LZ2WRQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.956879031 (5h57m26.543935272s) [noise=true , address=193.149.164.143:4698 , piece_id=AV2DCKYWWT5N4PL57WTXW45JNFYOCRLLEY6YSBSWOAQAPHAVXJQQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:13.036416979 (5h57m26.623473231s) [noise=true , address=104.36.82.9:4031 , piece_id=UOVVYSA77DQOK53ATDDXV56PLEVS6GBQ5HMZEKCEURYBPRCXNYPA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:13.172901316 (5h57m26.759957557s) [noise=true , address=104.36.82.47:4544 , piece_id=DVQRASYHRKPFEONZEWJ3MBVM7O5764NPWYADLMG4ARAH5PBD55MA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:13.202754351 (5h57m26.789810592s) [noise=true , address=104.36.82.14:4243 , piece_id=WKT5D56ZXVSJJHVR3GB6N5XR6CGU4VW2XK2YTMPBZOH4S2ZPOVCA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:13.202797742 (5h57m26.789853984s) [seg_pos={0 0}, upload=6089]: Segment reached optimal threshold of 14 pieces. Aug 7 22:24:13.203007333 (5h57m26.790063585s) [noise=true , address=104.36.82.58:4280 , piece_id=W2A65H4FIUOCUSMSVCANR2OPVBUFO4YNAFZV3KCASSCSPCHONGBA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fgHaEFKK4CeGg4x3vywTuYfT3kmac31xqg2QpsahQNXmvjhW8): protocol: expected piece hash; context canceled; EOF Aug 7 22:24:13.203190164 (5h57m26.790246415s) [noise=true , address=104.36.82.59:4333 , piece_id=JONL6XDWEUYMHVDPJVB54PE5HCYWN7KRO3FSNAMPJWCEP6QBKYVQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K7U6NXcB6SFBtQFrG7DXt2skACr3u5DX2QsrnToi8EKywS9TN): protocol: write tcp 5.161.89.252:63730->104.36.82.59:4333: use of closed network connection; write tcp 5.161.89.252:63730->104.36.82.59:4333: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:63730->104.36.82.59:4333: use of closed network connection Aug 7 22:24:13.203221153 (5h57m26.790277394s) [noise=true , address=104.36.82.45:4921 , piece_id=DCEBY7YTNAO4UR6YZ5LC5W27INFXZQ26BPJ7BHPMXY26P5BDL23Q, seg_pos={0 0}, upload=6089]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bBRKXiuSN2cw6KrdP8q4Dsf4MiFCUSgMEGjqG2dKL9RLTQra1): protocol: expected piece hash; context canceled; EOF Aug 7 22:24:13.203424783 (5h57m26.790481023s) [noise=true , address=104.36.82.62:4411 , piece_id=3TMITFZMXF5ESOCJNLFU6EIT7CFUYULYZ6FJUQ6X7T2WBEQMKWFQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DT68m6J4PpDk3XUDhDaaqV46NLKVz4A23716JLFxEAh122GeR): protocol: context canceled; context canceled Aug 7 22:24:13.203452675 (5h57m26.790508916s) [seg_pos={0 0}, upload=6089]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 22:24:13.203591695 (5h57m26.790647935s) [upload=6089]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:24:13.203602313 (5h57m26.790658565s) [upload=6089]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:29:03.852878645 (6h2m17.439934866s) [upload=6089]: Done uploading segments... Aug 7 22:29:03.852904053 (6h2m17.439960294s) [upload=6089]: Upload finished. err: metaclient: context deadline exceeded Duration: 1.722756738s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 21 out of 60 is failed (threshold=3) [................................XXXXXXXXXXXXXXXX.X.XXXX.....] duration: 180.308731ms FAIL (2025-08-07 22:32:12 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 180.337295ms core uplink us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 6 out of 60 is failed (threshold=3) [................................................XX.XXXX.....] duration: 21.022842ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 22:32:08 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 21.049522ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 24 out of 60 is failed (threshold=3) [.................................XXXXXXXXXXXXXXXXXXX..XXXX.X] duration: 30.000408167s FAIL (2025-08-07 22:32:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000427223s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 20 out of 60 is failed (threshold=3) [XXX.X.XX...............................XXXXXXXXXXX.X.XX.....] duration: 209.256044ms FAIL (2025-08-07 22:32:02 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 04]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:19:04.377299713 (5h52m17.964355924s) [upload=6018]: Starting upload Aug 7 22:19:04.377350166 (5h52m17.964406387s) [upload=6018]: Done starting upload Aug 7 22:19:04.377401523 (5h52m17.964457744s) [upload=6018]: Uploading segments... Aug 7 22:19:04.377446897 (5h52m17.964503118s) [upload=6018]: Got next segment. Inline: true Aug 7 22:19:04.377453410 (5h52m17.96450962s) [upload=6018]: Waiting for error group managing segments... Aug 7 22:19:04.377487142 (5h52m17.964543354s) [upload=6018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:19:04.377491361 (5h52m17.964547582s) [upload=6018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:19:04.377493875 (5h52m17.964550086s) [upload=6018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:20:04.361964803 (5h53m17.949021024s) [upload=6018]: Done uploading segments... Aug 7 22:20:04.361979240 (5h53m17.949035461s) [upload=6018]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:20:11.847632631 (5h53m25.434688843s) [upload=6035]: Starting upload Aug 7 22:20:11.847694537 (5h53m25.434750747s) [upload=6035]: Done starting upload Aug 7 22:20:11.847740192 (5h53m25.434796403s) [upload=6035]: Uploading segments... Aug 7 22:20:11.847772231 (5h53m25.434828442s) [upload=6035]: Got next segment. Inline: true Aug 7 22:20:11.847776860 (5h53m25.434833071s) [upload=6035]: Waiting for error group managing segments... Aug 7 22:20:11.847824679 (5h53m25.434880891s) [upload=6035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:20:11.847830130 (5h53m25.434886341s) [upload=6035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:20:11.847833205 (5h53m25.434889417s) [upload=6035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:21:11.836000640 (5h54m25.42305688s) [upload=6035]: Done uploading segments... Aug 7 22:21:11.836026087 (5h54m25.423082328s) [upload=6035]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:21:11.965120248 (5h54m25.552176489s) [upload=6050]: Starting upload Aug 7 22:21:11.965231515 (5h54m25.552287757s) [upload=6050]: Done starting upload Aug 7 22:21:11.965301156 (5h54m25.552357397s) [upload=6050]: Uploading segments... Aug 7 22:21:11.965348053 (5h54m25.552404304s) [upload=6050]: Got next segment. Inline: true Aug 7 22:21:11.965353994 (5h54m25.552410235s) [upload=6050]: Waiting for error group managing segments... Aug 7 22:21:11.965399049 (5h54m25.55245529s) [upload=6050]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:21:11.965405170 (5h54m25.552461411s) [upload=6050]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:21:11.965408957 (5h54m25.552465198s) [upload=6050]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:22:11.952333050 (5h55m25.539389261s) [upload=6050]: Done uploading segments... Aug 7 22:22:11.952348108 (5h55m25.539404329s) [upload=6050]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:22:12.125652873 (5h55m25.712709094s) [upload=6063]: Starting upload Aug 7 22:22:12.125702476 (5h55m25.712758688s) [upload=6063]: Done starting upload Aug 7 22:22:12.125747020 (5h55m25.712803241s) [upload=6063]: Uploading segments... Aug 7 22:22:12.125776895 (5h55m25.712833116s) [upload=6063]: Got next segment. Inline: true Aug 7 22:22:12.125781404 (5h55m25.712837625s) [upload=6063]: Waiting for error group managing segments... Aug 7 22:22:12.125814315 (5h55m25.712870536s) [upload=6063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:22:12.125818052 (5h55m25.712874263s) [upload=6063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:22:12.125820397 (5h55m25.712876618s) [upload=6063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:23:12.024254277 (5h56m25.611310498s) [upload=6063]: Done uploading segments... Aug 7 22:23:12.024289072 (5h56m25.611345283s) [upload=6063]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:23:12.175471355 (5h56m25.762527616s) [upload=6082]: Starting upload Aug 7 22:23:12.175573175 (5h56m25.762629466s) [upload=6082]: Done starting upload Aug 7 22:23:12.175670356 (5h56m25.762726627s) [upload=6082]: Uploading segments... Aug 7 22:23:12.175760093 (5h56m25.762816364s) [upload=6082]: Got next segment. Inline: true Aug 7 22:23:12.175767677 (5h56m25.762823919s) [upload=6082]: Waiting for error group managing segments... Aug 7 22:23:12.175819345 (5h56m25.762875585s) [upload=6082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:23:12.175825105 (5h56m25.762881346s) [upload=6082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:23:12.175828732 (5h56m25.762884973s) [upload=6082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:24:12.154559214 (5h57m25.741615455s) [upload=6082]: Done uploading segments... Aug 7 22:24:12.154578851 (5h57m25.741635092s) [upload=6082]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:24:12.253523032 (5h57m25.840579253s) [upload=6098]: Starting upload Aug 7 22:24:12.253572454 (5h57m25.840628665s) [upload=6098]: Done starting upload Aug 7 22:24:12.253615034 (5h57m25.840671245s) [upload=6098]: Uploading segments... Aug 7 22:24:12.253654808 (5h57m25.840711029s) [upload=6098]: Got next segment. Inline: true Aug 7 22:24:12.253658636 (5h57m25.840714847s) [upload=6098]: Waiting for error group managing segments... Aug 7 22:24:12.253689122 (5h57m25.840745343s) [upload=6098]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:24:12.253692969 (5h57m25.84074918s) [upload=6098]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:12.253695303 (5h57m25.840751524s) [upload=6098]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:24:12.467343237 (5h57m26.054399448s) [upload=6098]: Done uploading segments... Aug 7 22:24:12.467365428 (5h57m26.05442164s) [upload=6098]: Upload finished. err: <nil> Aug 7 22:24:33.085790933 (5h57m46.672847184s) [upload=6106]: Starting upload Aug 7 22:24:33.085870281 (5h57m46.672926522s) [upload=6106]: Done starting upload Aug 7 22:24:33.085932668 (5h57m46.67298892s) [upload=6106]: Uploading segments... Aug 7 22:24:33.085994053 (5h57m46.673050293s) [upload=6106]: Got next segment. Inline: true Aug 7 22:24:33.086000064 (5h57m46.673056304s) [upload=6106]: Waiting for error group managing segments... Aug 7 22:24:33.086045709 (5h57m46.67310195s) [upload=6106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:24:33.086051168 (5h57m46.67310741s) [upload=6106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:33.086054985 (5h57m46.673111227s) [upload=6106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:33.072711385 (5h58m46.659767636s) [upload=6106]: Done uploading segments... Aug 7 22:25:33.072753384 (5h58m46.659809634s) [upload=6106]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:25:33.160012555 (5h58m46.747068795s) [upload=6120]: Starting upload Aug 7 22:25:33.160101981 (5h58m46.747158222s) [upload=6120]: Done starting upload Aug 7 22:25:33.160199473 (5h58m46.747255715s) [upload=6120]: Uploading segments... Aug 7 22:25:33.160243596 (5h58m46.747299837s) [upload=6120]: Got next segment. Inline: true Aug 7 22:25:33.160249046 (5h58m46.747305287s) [upload=6120]: Waiting for error group managing segments... Aug 7 22:25:33.160294231 (5h58m46.747350482s) [upload=6120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:25:33.160300012 (5h58m46.747356252s) [upload=6120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:25:33.160303619 (5h58m46.747359859s) [upload=6120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:33.228188865 (5h58m46.815245086s) [upload=6120]: Done uploading segments... Aug 7 22:25:33.228202571 (5h58m46.815258782s) [upload=6120]: Upload finished. err: <nil> Aug 7 22:26:14.972993755 (5h59m28.560049967s) [upload=6132]: Starting upload Aug 7 22:26:14.973047095 (5h59m28.560103307s) [upload=6132]: Done starting upload Aug 7 22:26:14.973101456 (5h59m28.560157677s) [upload=6132]: Uploading segments... Aug 7 22:26:14.973137704 (5h59m28.560193925s) [upload=6132]: Got next segment. Inline: true Aug 7 22:26:14.973141382 (5h59m28.560197592s) [upload=6132]: Waiting for error group managing segments... Aug 7 22:26:14.973187948 (5h59m28.560244158s) [upload=6132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:26:14.973191995 (5h59m28.560248217s) [upload=6132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:26:14.973195201 (5h59m28.560251423s) [upload=6132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:14.950324988 (6h0m28.537381229s) [upload=6132]: Done uploading segments... Aug 7 22:27:14.950345787 (6h0m28.537402028s) [upload=6132]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:27:22.925128081 (6h0m36.512184292s) [upload=6146]: Starting upload Aug 7 22:27:22.925194244 (6h0m36.512250465s) [upload=6146]: Done starting upload Aug 7 22:27:22.925235571 (6h0m36.512291783s) [upload=6146]: Uploading segments... Aug 7 22:27:22.925260508 (6h0m36.512316729s) [upload=6146]: Got next segment. Inline: true Aug 7 22:27:22.925264486 (6h0m36.512320707s) [upload=6146]: Waiting for error group managing segments... Aug 7 22:27:22.925305752 (6h0m36.512361973s) [upload=6146]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:22.925309750 (6h0m36.512365971s) [upload=6146]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:22.925312044 (6h0m36.512368265s) [upload=6146]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:28:22.913115749 (6h1m36.50017199s) [upload=6146]: Done uploading segments... Aug 7 22:28:22.913136929 (6h1m36.500193169s) [upload=6146]: Upload finished. err: metaclient: context deadline exceeded Duration: 209.276702ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 20 out of 60 is failed (threshold=3) [XXX....................................XXXXXXXXXXXXXXXX..X..] duration: 17.277489867s FAIL (2025-08-07 22:31:58 UTC) operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 17.277512639s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 22 out of 60 is failed (threshold=3) [...X..............................XXXXXXXXXXXXXXXXX..XX..XX.] duration: 211.411558ms FAIL (2025-08-07 22:31:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 211.431635ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 26 out of 60 is failed (threshold=3) [XXXX.............................XXXXXXXXXXXXXXXX.XXXXX.X...] duration: 166.456052ms FAIL (2025-08-07 22:31:50 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 420501s) [upload=6055]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:22:06.462569787 (5h55m20.049626008s) [upload=6061]: Starting upload Aug 7 22:22:06.462655577 (5h55m20.049711788s) [upload=6061]: Done starting upload Aug 7 22:22:06.462750936 (5h55m20.049807157s) [upload=6061]: Uploading segments... Aug 7 22:22:06.462782464 (5h55m20.049838675s) [upload=6061]: Got next segment. Inline: true Aug 7 22:22:06.462786181 (5h55m20.049842402s) [upload=6061]: Waiting for error group managing segments... Aug 7 22:22:06.462817791 (5h55m20.049874001s) [upload=6061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:22:06.462821648 (5h55m20.049877869s) [upload=6061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:22:06.462823982 (5h55m20.049880203s) [upload=6061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:22:36.449657257 (5h55m50.036713498s) [upload=6061]: Done uploading segments... Aug 7 22:22:36.449678196 (5h55m50.036734448s) [upload=6061]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:22:59.911739779 (5h56m13.498795989s) [upload=6078]: Starting upload Aug 7 22:22:59.911786736 (5h56m13.498842957s) [upload=6078]: Done starting upload Aug 7 22:22:59.911827472 (5h56m13.498883683s) [upload=6078]: Uploading segments... Aug 7 22:22:59.911861074 (5h56m13.498917295s) [upload=6078]: Got next segment. Inline: true Aug 7 22:22:59.911864401 (5h56m13.498920622s) [upload=6078]: Waiting for error group managing segments... Aug 7 22:22:59.911889107 (5h56m13.498945319s) [upload=6078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:22:59.911893005 (5h56m13.498949226s) [upload=6078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:22:59.911894898 (5h56m13.498951119s) [upload=6078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:23:29.900458747 (5h56m43.487514958s) [upload=6078]: Done uploading segments... Aug 7 22:23:29.900473835 (5h56m43.487530047s) [upload=6078]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:23:58.370463254 (5h57m11.957519465s) [upload=6094]: Starting upload Aug 7 22:23:58.370512907 (5h57m11.957569128s) [upload=6094]: Done starting upload Aug 7 22:23:58.370553031 (5h57m11.957609243s) [upload=6094]: Uploading segments... Aug 7 22:23:58.370578639 (5h57m11.95763486s) [upload=6094]: Got next segment. Inline: true Aug 7 22:23:58.370582427 (5h57m11.957638648s) [upload=6094]: Waiting for error group managing segments... Aug 7 22:23:58.370610951 (5h57m11.957667172s) [upload=6094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:23:58.370615178 (5h57m11.957671399s) [upload=6094]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:23:58.370617392 (5h57m11.957673613s) [upload=6094]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:23:58.450972793 (5h57m12.038029004s) [upload=6094]: Done uploading segments... Aug 7 22:23:58.450986078 (5h57m12.038042289s) [upload=6094]: Upload finished. err: <nil> Aug 7 22:24:40.794473113 (5h57m54.381529325s) [upload=6107]: Starting upload Aug 7 22:24:40.794523428 (5h57m54.381579639s) [upload=6107]: Done starting upload Aug 7 22:24:40.794565206 (5h57m54.381621417s) [upload=6107]: Uploading segments... Aug 7 22:24:40.794591815 (5h57m54.381648026s) [upload=6107]: Got next segment. Inline: true Aug 7 22:24:40.794595622 (5h57m54.381651833s) [upload=6107]: Waiting for error group managing segments... Aug 7 22:24:40.794624367 (5h57m54.381680577s) [upload=6107]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:24:40.794628323 (5h57m54.381684535s) [upload=6107]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:40.794630638 (5h57m54.381686849s) [upload=6107]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:10.774549311 (5h58m24.361605533s) [upload=6107]: Done uploading segments... Aug 7 22:25:10.774570019 (5h58m24.361626231s) [upload=6107]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:25:10.855666099 (5h58m24.442722339s) [upload=6114]: Starting upload Aug 7 22:25:10.855752680 (5h58m24.442808921s) [upload=6114]: Done starting upload Aug 7 22:25:10.855822740 (5h58m24.442878991s) [upload=6114]: Uploading segments... Aug 7 22:25:10.855866683 (5h58m24.442922923s) [upload=6114]: Got next segment. Inline: true Aug 7 22:25:10.855872564 (5h58m24.442928805s) [upload=6114]: Waiting for error group managing segments... Aug 7 22:25:10.855966008 (5h58m24.443022249s) [upload=6114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:25:10.855972971 (5h58m24.443029212s) [upload=6114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:25:10.855976568 (5h58m24.443032819s) [upload=6114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:40.842457902 (5h58m54.429514153s) [upload=6114]: Done uploading segments... Aug 7 22:25:40.842479021 (5h58m54.429535272s) [upload=6114]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:25:40.940807525 (5h58m54.527863766s) [upload=6122]: Starting upload Aug 7 22:25:40.940896992 (5h58m54.527953233s) [upload=6122]: Done starting upload Aug 7 22:25:40.940964388 (5h58m54.528020628s) [upload=6122]: Uploading segments... Aug 7 22:25:40.941010434 (5h58m54.528066675s) [upload=6122]: Got next segment. Inline: true Aug 7 22:25:40.941016375 (5h58m54.528072616s) [upload=6122]: Waiting for error group managing segments... Aug 7 22:25:40.941060899 (5h58m54.52811715s) [upload=6122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:25:40.941068091 (5h58m54.528124332s) [upload=6122]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:25:40.941073060 (5h58m54.528129311s) [upload=6122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:26:10.929528068 (5h59m24.516584319s) [upload=6122]: Done uploading segments... Aug 7 22:26:10.929548557 (5h59m24.516604797s) [upload=6122]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:26:11.008231659 (5h59m24.59528788s) [upload=6129]: Starting upload Aug 7 22:26:11.008286501 (5h59m24.595342713s) [upload=6129]: Done starting upload Aug 7 22:26:11.008333198 (5h59m24.595389419s) [upload=6129]: Uploading segments... Aug 7 22:26:11.008364838 (5h59m24.595421059s) [upload=6129]: Got next segment. Inline: true Aug 7 22:26:11.008368595 (5h59m24.595424816s) [upload=6129]: Waiting for error group managing segments... Aug 7 22:26:11.008399943 (5h59m24.595456164s) [upload=6129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:26:11.008403599 (5h59m24.59545981s) [upload=6129]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:26:11.008406565 (5h59m24.595462786s) [upload=6129]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:26:40.996305184 (5h59m54.583361385s) [upload=6129]: Done uploading segments... Aug 7 22:26:40.996323769 (5h59m54.58337998s) [upload=6129]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:27:14.829073707 (6h0m28.416129948s) [upload=6144]: Starting upload Aug 7 22:27:14.829153145 (6h0m28.416209386s) [upload=6144]: Done starting upload Aug 7 22:27:14.829247050 (6h0m28.416303291s) [upload=6144]: Uploading segments... Aug 7 22:27:14.829299899 (6h0m28.41635614s) [upload=6144]: Got next segment. Inline: true Aug 7 22:27:14.829306702 (6h0m28.416362943s) [upload=6144]: Waiting for error group managing segments... Aug 7 22:27:14.829350864 (6h0m28.416407105s) [upload=6144]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:14.829357416 (6h0m28.416413657s) [upload=6144]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:14.829361143 (6h0m28.416417384s) [upload=6144]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:44.808549878 (6h0m58.395606099s) [upload=6144]: Done uploading segments... Aug 7 22:27:44.808564877 (6h0m58.395621098s) [upload=6144]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:28:23.619901331 (6h1m37.206957573s) [upload=6162]: Starting upload Aug 7 22:28:23.620017008 (6h1m37.207073248s) [upload=6162]: Done starting upload Aug 7 22:28:23.620072551 (6h1m37.207128791s) [upload=6162]: Uploading segments... Aug 7 22:28:23.620125059 (6h1m37.2071813s) [upload=6162]: Got next segment. Inline: true Aug 7 22:28:23.620132432 (6h1m37.207188672s) [upload=6162]: Waiting for error group managing segments... Aug 7 22:28:23.620198335 (6h1m37.207254577s) [upload=6162]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:28:23.620204407 (6h1m37.207260647s) [upload=6162]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:28:23.620208153 (6h1m37.207264395s) [upload=6162]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:28:23.683041871 (6h1m37.270098112s) [upload=6162]: Done uploading segments... Aug 7 22:28:23.683055537 (6h1m37.270111778s) [upload=6162]: Upload finished. err: <nil> Aug 7 22:28:40.804037302 (6h1m54.391093523s) [upload=6165]: Starting upload Aug 7 22:28:40.804089460 (6h1m54.391145671s) [upload=6165]: Done starting upload Aug 7 22:28:40.804134294 (6h1m54.391190515s) [upload=6165]: Uploading segments... Aug 7 22:28:40.804181422 (6h1m54.391237632s) [upload=6165]: Got next segment. Inline: true Aug 7 22:28:40.804185379 (6h1m54.3912416s) [upload=6165]: Waiting for error group managing segments... Aug 7 22:28:40.804217198 (6h1m54.391273419s) [upload=6165]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:28:40.804220915 (6h1m54.391277127s) [upload=6165]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:28:40.804223370 (6h1m54.391279591s) [upload=6165]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:29:10.793781791 (6h2m24.380838023s) [upload=6165]: Done uploading segments... Aug 7 22:29:10.793803392 (6h2m24.380859603s) [upload=6165]: Upload finished. err: metaclient: context deadline exceeded Duration: 166.478255ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 131.875742ms FAIL (2025-08-07 22:31:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ine Aug 7 21:59:56.515336538 (5h33m10.10239275s) [upload=5708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:59:56.574152557 (5h33m10.161208778s) [upload=5708]: Done uploading segments... Aug 7 21:59:56.574171943 (5h33m10.161228164s) [upload=5708]: Upload finished. err: <nil> Aug 7 22:00:36.611785100 (5h33m50.19884134s) [upload=5716]: Starting upload Aug 7 22:00:36.611856203 (5h33m50.198912444s) [upload=5716]: Done starting upload Aug 7 22:00:36.611959786 (5h33m50.199016027s) [upload=5716]: Uploading segments... Aug 7 22:00:36.612003668 (5h33m50.199059918s) [upload=5716]: Got next segment. Inline: true Aug 7 22:00:36.612009668 (5h33m50.19906591s) [upload=5716]: Waiting for error group managing segments... Aug 7 22:00:36.612057077 (5h33m50.199113319s) [upload=5716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:00:36.612082536 (5h33m50.199138776s) [upload=5716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:00:36.612086222 (5h33m50.199142473s) [upload=5716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:00:36.884781084 (5h33m50.471837294s) [upload=5716]: Done uploading segments... Aug 7 22:00:36.884795611 (5h33m50.471851832s) [upload=5716]: Upload finished. err: <nil> Aug 7 22:01:47.555449092 (5h35m1.142505313s) [upload=5736]: Starting upload Aug 7 22:01:47.555491781 (5h35m1.142548002s) [upload=5736]: Done starting upload Aug 7 22:01:47.555562123 (5h35m1.142618344s) [upload=5736]: Uploading segments... Aug 7 22:01:47.555595826 (5h35m1.142652047s) [upload=5736]: Got next segment. Inline: true Aug 7 22:01:47.555599363 (5h35m1.142655584s) [upload=5736]: Waiting for error group managing segments... Aug 7 22:01:47.555631602 (5h35m1.142687823s) [upload=5736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:47.555635390 (5h35m1.142691611s) [upload=5736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:47.555637894 (5h35m1.142694115s) [upload=5736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:47.686110188 (5h35m1.273166429s) [upload=5736]: Done uploading segments... Aug 7 22:01:47.686151165 (5h35m1.273207406s) [upload=5736]: Upload finished. err: <nil> Aug 7 22:02:22.678566055 (5h35m36.265622267s) [upload=5746]: Starting upload Aug 7 22:02:22.678633091 (5h35m36.265689303s) [upload=5746]: Done starting upload Aug 7 22:02:22.678684407 (5h35m36.265740617s) [upload=5746]: Uploading segments... Aug 7 22:02:22.678721757 (5h35m36.265777967s) [upload=5746]: Got next segment. Inline: true Aug 7 22:02:22.678725724 (5h35m36.265781935s) [upload=5746]: Waiting for error group managing segments... Aug 7 22:02:22.678759457 (5h35m36.265815668s) [upload=5746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:02:22.678764947 (5h35m36.265821168s) [upload=5746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:02:22.678768565 (5h35m36.265824826s) [upload=5746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:02:22.971349697 (5h35m36.558405918s) [upload=5746]: Done uploading segments... Aug 7 22:02:22.971373742 (5h35m36.558429963s) [upload=5746]: Upload finished. err: <nil> Aug 7 22:03:08.907032448 (5h36m22.494088689s) [upload=5757]: Starting upload Aug 7 22:03:08.907091528 (5h36m22.494147769s) [upload=5757]: Done starting upload Aug 7 22:03:08.907172368 (5h36m22.4942286s) [upload=5757]: Uploading segments... Aug 7 22:03:08.907209458 (5h36m22.494265699s) [upload=5757]: Got next segment. Inline: true Aug 7 22:03:08.907214758 (5h36m22.494271s) [upload=5757]: Waiting for error group managing segments... Aug 7 22:03:08.907255936 (5h36m22.494312176s) [upload=5757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:03:08.907261175 (5h36m22.494317416s) [upload=5757]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:03:08.907265904 (5h36m22.494322135s) [upload=5757]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:03:09.004207779 (5h36m22.59126404s) [upload=5757]: Done uploading segments... Aug 7 22:03:09.004232375 (5h36m22.591288627s) [upload=5757]: Upload finished. err: <nil> Aug 7 22:04:02.463086309 (5h37m16.050142521s) [upload=5775]: Starting upload Aug 7 22:04:02.463128347 (5h37m16.050184568s) [upload=5775]: Done starting upload Aug 7 22:04:02.463197818 (5h37m16.050254029s) [upload=5775]: Uploading segments... Aug 7 22:04:02.463223595 (5h37m16.050279816s) [upload=5775]: Got next segment. Inline: true Aug 7 22:04:02.463227652 (5h37m16.050283873s) [upload=5775]: Waiting for error group managing segments... Aug 7 22:04:02.463258120 (5h37m16.050314331s) [upload=5775]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:04:02.463261796 (5h37m16.050318008s) [upload=5775]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:04:02.463264733 (5h37m16.050320954s) [upload=5775]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:04:02.530547234 (5h37m16.117603445s) [upload=5775]: Done uploading segments... Aug 7 22:04:02.530562512 (5h37m16.117618733s) [upload=5775]: Upload finished. err: <nil> Aug 7 22:05:10.744065032 (5h38m24.331121272s) [upload=5796]: Starting upload Aug 7 22:05:10.744195716 (5h38m24.331251957s) [upload=5796]: Done starting upload Aug 7 22:05:10.744327762 (5h38m24.331384002s) [upload=5796]: Uploading segments... Aug 7 22:05:10.744369549 (5h38m24.331425791s) [upload=5796]: Got next segment. Inline: true Aug 7 22:05:10.744388655 (5h38m24.331444897s) [upload=5796]: Waiting for error group managing segments... Aug 7 22:05:10.744449028 (5h38m24.331505259s) [upload=5796]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:05:10.744454398 (5h38m24.331510629s) [upload=5796]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:05:10.744457633 (5h38m24.331513875s) [upload=5796]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:05:10.852991578 (5h38m24.440047789s) [upload=5796]: Done uploading segments... Aug 7 22:05:10.853004571 (5h38m24.440060782s) [upload=5796]: Upload finished. err: <nil> Aug 7 22:05:45.060720865 (5h38m58.647777086s) [upload=5806]: Starting upload Aug 7 22:05:45.060780046 (5h38m58.647836267s) [upload=5806]: Done starting upload Aug 7 22:05:45.060842121 (5h38m58.647898333s) [upload=5806]: Uploading segments... Aug 7 22:05:45.060876836 (5h38m58.647933057s) [upload=5806]: Got next segment. Inline: true Aug 7 22:05:45.060882878 (5h38m58.647939099s) [upload=5806]: Waiting for error group managing segments... Aug 7 22:05:45.060927901 (5h38m58.647984112s) [upload=5806]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:05:45.060933843 (5h38m58.647990064s) [upload=5806]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:05:45.060936187 (5h38m58.647992408s) [upload=5806]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:05:45.328399453 (5h38m58.915455674s) [upload=5806]: Done uploading segments... Aug 7 22:05:45.328433367 (5h38m58.915489588s) [upload=5806]: Upload finished. err: <nil> Aug 7 22:06:12.931860875 (5h39m26.518917096s) [upload=5815]: Starting upload Aug 7 22:06:12.931905979 (5h39m26.51896219s) [upload=5815]: Done starting upload Aug 7 22:06:12.931964518 (5h39m26.51902073s) [upload=5815]: Uploading segments... Aug 7 22:06:12.931990617 (5h39m26.519046827s) [upload=5815]: Got next segment. Inline: true Aug 7 22:06:12.931994224 (5h39m26.519050434s) [upload=5815]: Waiting for error group managing segments... Aug 7 22:06:12.932024821 (5h39m26.519081042s) [upload=5815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:06:12.932028408 (5h39m26.519084618s) [upload=5815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:06:12.932030893 (5h39m26.519087103s) [upload=5815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:06:13.014995076 (5h39m26.602051317s) [upload=5815]: Done uploading segments... Aug 7 22:06:13.015014863 (5h39m26.602071114s) [upload=5815]: Upload finished. err: <nil> Aug 7 22:07:16.916734820 (5h40m30.503791062s) [upload=5832]: Starting upload Aug 7 22:07:16.916800323 (5h40m30.503856563s) [upload=5832]: Done starting upload Aug 7 22:07:16.916864883 (5h40m30.503921123s) [upload=5832]: Uploading segments... Aug 7 22:07:16.916903304 (5h40m30.503959546s) [upload=5832]: Got next segment. Inline: true Aug 7 22:07:16.916908815 (5h40m30.503965056s) [upload=5832]: Waiting for error group managing segments... Aug 7 22:07:16.916952356 (5h40m30.504008598s) [upload=5832]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:07:16.916957657 (5h40m30.504013897s) [upload=5832]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:07:16.916961754 (5h40m30.504017995s) [upload=5832]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:07:16.988776316 (5h40m30.575832566s) [upload=5832]: Done uploading segments... Aug 7 22:07:16.988801573 (5h40m30.575857824s) [upload=5832]: Upload finished. err: <nil> Aug 7 22:08:29.650294124 (5h41m43.237350365s) [upload=5854]: Starting upload Aug 7 22:08:29.650357383 (5h41m43.237413623s) [upload=5854]: Done starting upload Aug 7 22:08:29.650418325 (5h41m43.237474567s) [upload=5854]: Uploading segments... Aug 7 22:08:29.650468490 (5h41m43.237524731s) [upload=5854]: Got next segment. Inline: true Aug 7 22:08:29.650474350 (5h41m43.237530592s) [upload=5854]: Waiting for error group managing segments... Aug 7 22:08:29.650515618 (5h41m43.237571859s) [upload=5854]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:08:29.650522029 (5h41m43.237578271s) [upload=5854]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:08:29.650525556 (5h41m43.237581797s) [upload=5854]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:08:29.706715553 (5h41m43.293771793s) [upload=5854]: Done uploading segments... Aug 7 22:08:29.706731001 (5h41m43.293787243s) [upload=5854]: Upload finished. err: metaclient: Too Many Requests Duration:* 131.905318ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 116.335755ms remote: "tcp:136.0.77.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 22:31:46 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 116.38084ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 21 out of 60 is failed (threshold=3) [.................................XXXXXXXXXXXXXXXX.X.XXXX....] duration: 173.620396ms FAIL (2025-08-07 22:31:42 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 173.642848ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [...................................XXXXXXXXXXXXXX.XXX.......] duration: 388.267865ms FAIL (2025-08-07 22:31:36 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== TVRWLFHTOF2YZFDYX3LU5FENJAS6JOW572GLJ2OHVQ, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.749054955 (5h56m13.336111176s) [noise=true , address=104.36.82.2:4730 , piece_id=FKFZNBUC7L2WVTTFPNRSPJUWW6HAEE6CNUAYSHVDYXW5XENS462Q, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.749426248 (5h56m13.336482459s) [noise=true , address=45.91.116.9:4262 , piece_id=R7J2NVZJTXR7ATH2VUC75B2Z73TSJAQUBYJ3WE75RQ6AD3L3PSCQ, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.750681361 (5h56m13.337737582s) [noise=true , address=104.36.82.5:4840 , piece_id=6SYENOMYMIEYCEFSHJKMXOQE4RCDPDWBAHYDVRTXCQYWJGQDCCKA, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.751093369 (5h56m13.338149581s) [noise=true , address=104.36.82.14:4221 , piece_id=SBKEVRBOXXBFOVTZWPSDVDPXEDV6HZIVAV4MWFIYDMNEU643KBWA, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.751643326 (5h56m13.338699547s) [noise=true , address=45.91.116.4:4940 , piece_id=UXKWVJQC6KYUHGXKHRYHJATOP5KQF4BY2LWNQJQEGFYKQEBT6GWA, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.752249137 (5h56m13.339305348s) [noise=true , address=77.90.31.13:4628 , piece_id=BYQWTXNUD76XIWZZAUZNHDUX7F7CABMDCPK7M7QPDHS3CFBRBMVQ, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.752826465 (5h56m13.339882676s) [noise=true , address=104.36.82.2:4731 , piece_id=L6W56WDLHKGHEPXBK7F2OYZ6CRM66ZVKG5RGJPH3FHC7RZZ3QGBA, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.753473372 (5h56m13.340529583s) [noise=true , address=45.91.116.12:4227 , piece_id=7RIB2VCODFSHTRTBGI3Z74IECO3ATQXSXW7CY3TCA43MZKSKEQYA, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.753961884 (5h56m13.341018096s) [noise=true , address=77.90.31.10:4519 , piece_id=QF2SVPCWB5HG4S7GQL6NFHTYPVOHP4HHORRY2TMJSRXP5A3J43TQ, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.754616727 (5h56m13.341672948s) [noise=true , address=104.36.82.9:4006 , piece_id=MDXFW3M6TBQJXYHMBTPIOD5I5PK3NUHDRZ3EERNCRKFVMUPSW43Q, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.755122451 (5h56m13.342178672s) [noise=true , address=104.36.82.8:4971 , piece_id=2MZZBGEPYJSHOBAG4FUAU5NKKAOL72C2AWD3GZECQXGPMOG6TDMQ, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.837536198 (5h56m13.424592419s) [upload=6077]: Next returned nil segment Aug 7 22:22:59.837547418 (5h56m13.424603629s) [upload=6077]: Waiting for error group managing segments... Aug 7 22:22:59.846810959 (5h56m13.43386718s) [noise=true , address=45.91.116.7:4011 , piece_id=E7YGVXQXIO2A34B22AHGKBQXER4KLEX3AFGHVFUQ6USLQMUFSW2Q, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.847279324 (5h56m13.434335535s) [noise=true , address=45.91.116.12:4227 , piece_id=7RIB2VCODFSHTRTBGI3Z74IECO3ATQXSXW7CY3TCA43MZKSKEQYA, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.848559454 (5h56m13.435615675s) [noise=true , address=45.91.116.4:4940 , piece_id=UXKWVJQC6KYUHGXKHRYHJATOP5KQF4BY2LWNQJQEGFYKQEBT6GWA, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.852382881 (5h56m13.439439102s) [noise=true , address=45.91.116.9:4262 , piece_id=R7J2NVZJTXR7ATH2VUC75B2Z73TSJAQUBYJ3WE75RQ6AD3L3PSCQ, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.852577974 (5h56m13.439634195s) [noise=true , address=77.90.31.19:4842 , piece_id=ULCHXICRO3ULX7IEW6GVXHHGXT6F42N3WQDEEUVOSKCXID6F6XZQ, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.856642091 (5h56m13.443698302s) [noise=true , address=104.36.82.8:4971 , piece_id=2MZZBGEPYJSHOBAG4FUAU5NKKAOL72C2AWD3GZECQXGPMOG6TDMQ, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.856766534 (5h56m13.443822745s) [noise=true , address=77.90.31.10:4519 , piece_id=QF2SVPCWB5HG4S7GQL6NFHTYPVOHP4HHORRY2TMJSRXP5A3J43TQ, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.868085922 (5h56m13.455142143s) [noise=true , address=45.91.116.12:4225 , piece_id=MNJGBICBIG3HQXCO7B2C2ITRFSOW475RAGNVTAQNVFQYAJ7DOU5Q, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.868246762 (5h56m13.455302973s) [noise=true , address=104.36.82.2:4730 , piece_id=FKFZNBUC7L2WVTTFPNRSPJUWW6HAEE6CNUAYSHVDYXW5XENS462Q, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.877804453 (5h56m13.464860674s) [noise=true , address=104.36.82.11:4115 , piece_id=BALIRX6S5WW2CGSXSIFHWV2ABDH3ASPHR36YEYRUYTTGZCW6U6AQ, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.882357252 (5h56m13.469413462s) [noise=true , address=104.36.82.2:4731 , piece_id=L6W56WDLHKGHEPXBK7F2OYZ6CRM66ZVKG5RGJPH3FHC7RZZ3QGBA, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.887398722 (5h56m13.474454933s) [noise=true , address=104.36.82.14:4221 , piece_id=SBKEVRBOXXBFOVTZWPSDVDPXEDV6HZIVAV4MWFIYDMNEU643KBWA, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.887674137 (5h56m13.474730358s) [noise=true , address=104.36.82.10:4079 , piece_id=DHLJEMGMQUTVRWLFHTOF2YZFDYX3LU5FENJAS6JOW572GLJ2OHVQ, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.887689665 (5h56m13.474745877s) [seg_pos={0 0}, upload=6077]: Segment reached optimal threshold of 13 pieces. Aug 7 22:22:59.887775385 (5h56m13.474831606s) [noise=true , address=77.90.31.13:4628 , piece_id=BYQWTXNUD76XIWZZAUZNHDUX7F7CABMDCPK7M7QPDHS3CFBRBMVQ, seg_pos={0 0}, upload=6077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: expected piece hash; context canceled; EOF Aug 7 22:22:59.887795453 (5h56m13.474851674s) [noise=true , address=104.36.82.5:4840 , piece_id=6SYENOMYMIEYCEFSHJKMXOQE4RCDPDWBAHYDVRTXCQYWJGQDCCKA, seg_pos={0 0}, upload=6077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Aug 7 22:22:59.887879469 (5h56m13.47493569s) [noise=true , address=104.36.82.9:4006 , piece_id=MDXFW3M6TBQJXYHMBTPIOD5I5PK3NUHDRZ3EERNCRKFVMUPSW43Q, seg_pos={0 0}, upload=6077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc): protocol: write tcp 5.161.89.252:59712->104.36.82.9:4006: use of closed network connection; write tcp 5.161.89.252:59712->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:59712->104.36.82.9:4006: use of closed network connection Aug 7 22:22:59.887943669 (5h56m13.47499988s) [seg_pos={0 0}, upload=6077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 22:22:59.888006888 (5h56m13.475063109s) [upload=6077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:22:59.888010665 (5h56m13.475066886s) [upload=6077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:22:59.933305033 (5h56m13.520361243s) [upload=6077]: Done uploading segments... Aug 7 22:22:59.933317746 (5h56m13.520373958s) [upload=6077]: Upload finished. err: <nil> Aug 7 22:23:47.739190890 (5h57m1.326247131s) [upload=6091]: Starting upload Aug 7 22:23:47.739297178 (5h57m1.326353419s) [upload=6091]: Done starting upload Aug 7 22:23:47.739428053 (5h57m1.326484274s) [upload=6091]: Uploading segments... Aug 7 22:23:47.739777785 (5h57m1.326834025s) [upload=6091]: Got next segment. Inline: false Aug 7 22:23:47.739805196 (5h57m1.326861407s) [upload=6091]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:23:47.739811638 (5h57m1.326867849s) [upload=6091]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:24:17.722308743 (5h57m31.309364985s) [upload=6091]: Next segment err: context deadline exceeded Aug 7 22:24:17.722438996 (5h57m31.309495238s) [upload=6091]: Done uploading segments... Aug 7 22:24:17.722476005 (5h57m31.309532246s) [upload=6091]: Upload finished. err: context deadline exceeded Aug 7 22:24:18.676382471 (5h57m32.263438711s) [upload=6101]: Starting upload Aug 7 22:24:18.676482739 (5h57m32.263538979s) [upload=6101]: Done starting upload Aug 7 22:24:18.676619723 (5h57m32.263675964s) [upload=6101]: Uploading segments... Aug 7 22:24:18.676886642 (5h57m32.263942883s) [upload=6101]: Got next segment. Inline: false Aug 7 22:24:18.676922940 (5h57m32.263979181s) [upload=6101]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:24:18.676930835 (5h57m32.263987076s) [upload=6101]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:24:48.662510024 (5h58m2.249566255s) [upload=6101]: Next segment err: context deadline exceeded Aug 7 22:24:48.662708235 (5h58m2.249764465s) [upload=6101]: Done uploading segments... Aug 7 22:24:48.662735284 (5h58m2.249791526s) [upload=6101]: Upload finished. err: context deadline exceeded Aug 7 22:25:18.142081018 (5h58m31.729137259s) [upload=6117]: Starting upload Aug 7 22:25:18.142160586 (5h58m31.729216828s) [upload=6117]: Done starting upload Aug 7 22:25:18.142286381 (5h58m31.729342623s) [upload=6117]: Uploading segments... Aug 7 22:25:18.142594587 (5h58m31.729650828s) [upload=6117]: Got next segment. Inline: false Aug 7 22:25:18.142641995 (5h58m31.729698236s) [upload=6117]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:25:18.142647436 (5h58m31.729703676s) [upload=6117]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:25:48.125202589 (5h59m1.71225884s) [upload=6117]: Next segment err: context deadline exceeded Aug 7 22:25:48.125308446 (5h59m1.712364688s) [upload=6117]: Done uploading segments... Aug 7 22:25:48.125355273 (5h59m1.712411525s) [upload=6117]: Upload finished. err: context deadline exceeded Duration: 388.294385ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 21 out of 60 is failed (threshold=3) [XXXXXXXXXXXX..................................XXXXXXXX.X....] duration: 1.731812732s FAIL (2025-08-07 22:31:36 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.73183883s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 20 out of 60 is failed (threshold=3) [XXXX.XX......X.............................XXXXXXXXXXXXX....] duration: 1.18293152s FAIL (2025-08-07 22:31:36 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== [upload=6089]: Done starting upload Aug 7 22:23:43.961837605 (5h56m57.548893826s) [upload=6089]: Uploading segments... Aug 7 22:23:43.962138106 (5h56m57.549194347s) [upload=6089]: Got next segment. Inline: false Aug 7 22:23:43.962204581 (5h56m57.549260822s) [upload=6089]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:23:43.962209980 (5h56m57.549266222s) [upload=6089]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:24:12.375269699 (5h57m25.96232592s) [seg_pos={0 0}, upload=6089]: Begin upload segment... Aug 7 22:24:12.375313250 (5h57m25.962369471s) [seg_pos={0 0}, upload=6089]: Done begin upload segment. Aug 7 22:24:12.375339780 (5h57m25.962396031s) [noise=true , address=193.149.164.143:4698 , piece_id=AV2DCKYWWT5N4PL57WTXW45JNFYOCRLLEY6YSBSWOAQAPHAVXJQQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.375356050 (5h57m25.962412261s) [noise=true , address=104.36.82.53:4156 , piece_id=NBWZ4FB4S7CTDN2OTWTDKSH2R7DJN66P7GRTQTQ5FJZFQCZLDAQQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.375715481 (5h57m25.962771693s) [noise=true , address=104.36.82.58:4280 , piece_id=W2A65H4FIUOCUSMSVCANR2OPVBUFO4YNAFZV3KCASSCSPCHONGBA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.375724769 (5h57m25.96278099s) [noise=true , address=104.36.82.59:4333 , piece_id=JONL6XDWEUYMHVDPJVB54PE5HCYWN7KRO3FSNAMPJWCEP6QBKYVQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376022925 (5h57m25.963079146s) [noise=true , address=45.91.116.9:4270 , piece_id=CMJ7K7XDV44UUOWXU6NOUNMWZGLOEZKPTLVFMTNWIR6OK52TOEKQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376075263 (5h57m25.963131475s) [noise=true , address=104.36.82.36:4652 , piece_id=ZDJ2LSAX25MBAAW4PW745SX7J7X3S4PAGXEUS36T43KTM3LZ2WRQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376490167 (5h57m25.963546388s) [noise=true , address=104.36.82.45:4921 , piece_id=DCEBY7YTNAO4UR6YZ5LC5W27INFXZQ26BPJ7BHPMXY26P5BDL23Q, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376527327 (5h57m25.963583599s) [noise=true , address=104.36.82.12:4169 , piece_id=XPNXIWBPTKCEEQTVQ3BAQ3HCLQE5U5BTRIAR3FQ6O4OQJZMCJ6ZA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376755913 (5h57m25.963812125s) [noise=true , address=104.36.82.62:4411 , piece_id=3TMITFZMXF5ESOCJNLFU6EIT7CFUYULYZ6FJUQ6X7T2WBEQMKWFQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376864006 (5h57m25.963920216s) [noise=true , address=104.36.82.14:4243 , piece_id=WKT5D56ZXVSJJHVR3GB6N5XR6CGU4VW2XK2YTMPBZOH4S2ZPOVCA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376979641 (5h57m25.964035862s) [noise=true , address=104.36.82.9:4031 , piece_id=UOVVYSA77DQOK53ATDDXV56PLEVS6GBQ5HMZEKCEURYBPRCXNYPA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.377222374 (5h57m25.964278595s) [noise=true , address=104.36.82.52:4100 , piece_id=7RUPILCVG3GL3KB4CS25YCYKDB4C6HFT6TGZWZIDC5EHBI2SD55Q, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.377246499 (5h57m25.96430271s) [noise=true , address=79.127.201.215:4343 , piece_id=N3KGRRGXWJLCJGYSLPOLOIYZHP7AFWV6VBCHVBETDYOQBIXIWPMQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.377620267 (5h57m25.964676488s) [noise=true , address=104.36.82.35:4597 , piece_id=6UMSPJQOVLEDWMIQMLAJLZ2AY6Y5XLMODCN3V63EB5JNJS3EC7BA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.377642138 (5h57m25.964698349s) [noise=true , address=104.36.82.16:4300 , piece_id=RJRU7SETYXWMQK7GEE3HBD2XMEVSI3UJ6ZDCIWVW4266QSNAOPZA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.377932059 (5h57m25.96498828s) [noise=true , address=104.36.82.47:4544 , piece_id=DVQRASYHRKPFEONZEWJ3MBVM7O5764NPWYADLMG4ARAH5PBD55MA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.377950764 (5h57m25.965006985s) [noise=true , address=104.36.82.21:4478 , piece_id=5UO756WUOKCS52UJI2NIQFUSU7W6NBW34MGFH6YZX27HKWY5243A, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.378280479 (5h57m25.9653367s) [noise=true , address=104.36.82.37:4656 , piece_id=A25QOY4UFT7RRRY6PVAS3NNML7PGJM4OHDH56PF2HZHKJ7VOPURA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.468526485 (5h57m26.055582696s) [upload=6089]: Next returned nil segment Aug 7 22:24:12.468535081 (5h57m26.055591292s) [upload=6089]: Waiting for error group managing segments... Aug 7 22:24:12.483383738 (5h57m26.070439949s) [noise=true , address=104.36.82.37:4656 , piece_id=A25QOY4UFT7RRRY6PVAS3NNML7PGJM4OHDH56PF2HZHKJ7VOPURA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.500821799 (5h57m26.08787801s) [noise=true , address=104.36.82.16:4300 , piece_id=RJRU7SETYXWMQK7GEE3HBD2XMEVSI3UJ6ZDCIWVW4266QSNAOPZA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.504410257 (5h57m26.091466458s) [noise=true , address=45.91.116.9:4270 , piece_id=CMJ7K7XDV44UUOWXU6NOUNMWZGLOEZKPTLVFMTNWIR6OK52TOEKQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.511708921 (5h57m26.098765132s) [noise=true , address=104.36.82.35:4597 , piece_id=6UMSPJQOVLEDWMIQMLAJLZ2AY6Y5XLMODCN3V63EB5JNJS3EC7BA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.523758022 (5h57m26.110814233s) [noise=true , address=104.36.82.12:4169 , piece_id=XPNXIWBPTKCEEQTVQ3BAQ3HCLQE5U5BTRIAR3FQ6O4OQJZMCJ6ZA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.533614250 (5h57m26.120670461s) [noise=true , address=104.36.82.52:4100 , piece_id=7RUPILCVG3GL3KB4CS25YCYKDB4C6HFT6TGZWZIDC5EHBI2SD55Q, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.557032764 (5h57m26.144088985s) [noise=true , address=104.36.82.21:4478 , piece_id=5UO756WUOKCS52UJI2NIQFUSU7W6NBW34MGFH6YZX27HKWY5243A, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.582698011 (5h57m26.169754223s) [noise=true , address=104.36.82.53:4156 , piece_id=NBWZ4FB4S7CTDN2OTWTDKSH2R7DJN66P7GRTQTQ5FJZFQCZLDAQQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.609343038 (5h57m26.196399279s) [noise=true , address=79.127.201.215:4343 , piece_id=N3KGRRGXWJLCJGYSLPOLOIYZHP7AFWV6VBCHVBETDYOQBIXIWPMQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.639134328 (5h57m26.226190538s) [noise=true , address=104.36.82.36:4652 , piece_id=ZDJ2LSAX25MBAAW4PW745SX7J7X3S4PAGXEUS36T43KTM3LZ2WRQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.956879031 (5h57m26.543935272s) [noise=true , address=193.149.164.143:4698 , piece_id=AV2DCKYWWT5N4PL57WTXW45JNFYOCRLLEY6YSBSWOAQAPHAVXJQQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:13.036416979 (5h57m26.623473231s) [noise=true , address=104.36.82.9:4031 , piece_id=UOVVYSA77DQOK53ATDDXV56PLEVS6GBQ5HMZEKCEURYBPRCXNYPA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:13.172901316 (5h57m26.759957557s) [noise=true , address=104.36.82.47:4544 , piece_id=DVQRASYHRKPFEONZEWJ3MBVM7O5764NPWYADLMG4ARAH5PBD55MA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:13.202754351 (5h57m26.789810592s) [noise=true , address=104.36.82.14:4243 , piece_id=WKT5D56ZXVSJJHVR3GB6N5XR6CGU4VW2XK2YTMPBZOH4S2ZPOVCA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:13.202797742 (5h57m26.789853984s) [seg_pos={0 0}, upload=6089]: Segment reached optimal threshold of 14 pieces. Aug 7 22:24:13.203007333 (5h57m26.790063585s) [noise=true , address=104.36.82.58:4280 , piece_id=W2A65H4FIUOCUSMSVCANR2OPVBUFO4YNAFZV3KCASSCSPCHONGBA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fgHaEFKK4CeGg4x3vywTuYfT3kmac31xqg2QpsahQNXmvjhW8): protocol: expected piece hash; context canceled; EOF Aug 7 22:24:13.203190164 (5h57m26.790246415s) [noise=true , address=104.36.82.59:4333 , piece_id=JONL6XDWEUYMHVDPJVB54PE5HCYWN7KRO3FSNAMPJWCEP6QBKYVQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K7U6NXcB6SFBtQFrG7DXt2skACr3u5DX2QsrnToi8EKywS9TN): protocol: write tcp 5.161.89.252:63730->104.36.82.59:4333: use of closed network connection; write tcp 5.161.89.252:63730->104.36.82.59:4333: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:63730->104.36.82.59:4333: use of closed network connection Aug 7 22:24:13.203221153 (5h57m26.790277394s) [noise=true , address=104.36.82.45:4921 , piece_id=DCEBY7YTNAO4UR6YZ5LC5W27INFXZQ26BPJ7BHPMXY26P5BDL23Q, seg_pos={0 0}, upload=6089]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bBRKXiuSN2cw6KrdP8q4Dsf4MiFCUSgMEGjqG2dKL9RLTQra1): protocol: expected piece hash; context canceled; EOF Aug 7 22:24:13.203424783 (5h57m26.790481023s) [noise=true , address=104.36.82.62:4411 , piece_id=3TMITFZMXF5ESOCJNLFU6EIT7CFUYULYZ6FJUQ6X7T2WBEQMKWFQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DT68m6J4PpDk3XUDhDaaqV46NLKVz4A23716JLFxEAh122GeR): protocol: context canceled; context canceled Aug 7 22:24:13.203452675 (5h57m26.790508916s) [seg_pos={0 0}, upload=6089]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 22:24:13.203591695 (5h57m26.790647935s) [upload=6089]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:24:13.203602313 (5h57m26.790658565s) [upload=6089]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:29:03.852878645 (6h2m17.439934866s) [upload=6089]: Done uploading segments... Aug 7 22:29:03.852904053 (6h2m17.439960294s) [upload=6089]: Upload finished. err: metaclient: context deadline exceeded Duration: 1.182951468s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 22 out of 60 is failed (threshold=3) [....X..............................XXXXXXXXXXXXXXXXX..XX..XX] duration: 30.000942813s FAIL (2025-08-07 22:31:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000990531s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 6 out of 60 is failed (threshold=3) [.................................................XX.XXXX....] duration: 29.038735ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 22:31:26 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 29.056488ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 238.269021ms FAIL (2025-08-07 22:31:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ine Aug 7 21:59:56.515336538 (5h33m10.10239275s) [upload=5708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:59:56.574152557 (5h33m10.161208778s) [upload=5708]: Done uploading segments... Aug 7 21:59:56.574171943 (5h33m10.161228164s) [upload=5708]: Upload finished. err: <nil> Aug 7 22:00:36.611785100 (5h33m50.19884134s) [upload=5716]: Starting upload Aug 7 22:00:36.611856203 (5h33m50.198912444s) [upload=5716]: Done starting upload Aug 7 22:00:36.611959786 (5h33m50.199016027s) [upload=5716]: Uploading segments... Aug 7 22:00:36.612003668 (5h33m50.199059918s) [upload=5716]: Got next segment. Inline: true Aug 7 22:00:36.612009668 (5h33m50.19906591s) [upload=5716]: Waiting for error group managing segments... Aug 7 22:00:36.612057077 (5h33m50.199113319s) [upload=5716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:00:36.612082536 (5h33m50.199138776s) [upload=5716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:00:36.612086222 (5h33m50.199142473s) [upload=5716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:00:36.884781084 (5h33m50.471837294s) [upload=5716]: Done uploading segments... Aug 7 22:00:36.884795611 (5h33m50.471851832s) [upload=5716]: Upload finished. err: <nil> Aug 7 22:01:47.555449092 (5h35m1.142505313s) [upload=5736]: Starting upload Aug 7 22:01:47.555491781 (5h35m1.142548002s) [upload=5736]: Done starting upload Aug 7 22:01:47.555562123 (5h35m1.142618344s) [upload=5736]: Uploading segments... Aug 7 22:01:47.555595826 (5h35m1.142652047s) [upload=5736]: Got next segment. Inline: true Aug 7 22:01:47.555599363 (5h35m1.142655584s) [upload=5736]: Waiting for error group managing segments... Aug 7 22:01:47.555631602 (5h35m1.142687823s) [upload=5736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:47.555635390 (5h35m1.142691611s) [upload=5736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:47.555637894 (5h35m1.142694115s) [upload=5736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:47.686110188 (5h35m1.273166429s) [upload=5736]: Done uploading segments... Aug 7 22:01:47.686151165 (5h35m1.273207406s) [upload=5736]: Upload finished. err: <nil> Aug 7 22:02:22.678566055 (5h35m36.265622267s) [upload=5746]: Starting upload Aug 7 22:02:22.678633091 (5h35m36.265689303s) [upload=5746]: Done starting upload Aug 7 22:02:22.678684407 (5h35m36.265740617s) [upload=5746]: Uploading segments... Aug 7 22:02:22.678721757 (5h35m36.265777967s) [upload=5746]: Got next segment. Inline: true Aug 7 22:02:22.678725724 (5h35m36.265781935s) [upload=5746]: Waiting for error group managing segments... Aug 7 22:02:22.678759457 (5h35m36.265815668s) [upload=5746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:02:22.678764947 (5h35m36.265821168s) [upload=5746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:02:22.678768565 (5h35m36.265824826s) [upload=5746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:02:22.971349697 (5h35m36.558405918s) [upload=5746]: Done uploading segments... Aug 7 22:02:22.971373742 (5h35m36.558429963s) [upload=5746]: Upload finished. err: <nil> Aug 7 22:03:08.907032448 (5h36m22.494088689s) [upload=5757]: Starting upload Aug 7 22:03:08.907091528 (5h36m22.494147769s) [upload=5757]: Done starting upload Aug 7 22:03:08.907172368 (5h36m22.4942286s) [upload=5757]: Uploading segments... Aug 7 22:03:08.907209458 (5h36m22.494265699s) [upload=5757]: Got next segment. Inline: true Aug 7 22:03:08.907214758 (5h36m22.494271s) [upload=5757]: Waiting for error group managing segments... Aug 7 22:03:08.907255936 (5h36m22.494312176s) [upload=5757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:03:08.907261175 (5h36m22.494317416s) [upload=5757]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:03:08.907265904 (5h36m22.494322135s) [upload=5757]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:03:09.004207779 (5h36m22.59126404s) [upload=5757]: Done uploading segments... Aug 7 22:03:09.004232375 (5h36m22.591288627s) [upload=5757]: Upload finished. err: <nil> Aug 7 22:04:02.463086309 (5h37m16.050142521s) [upload=5775]: Starting upload Aug 7 22:04:02.463128347 (5h37m16.050184568s) [upload=5775]: Done starting upload Aug 7 22:04:02.463197818 (5h37m16.050254029s) [upload=5775]: Uploading segments... Aug 7 22:04:02.463223595 (5h37m16.050279816s) [upload=5775]: Got next segment. Inline: true Aug 7 22:04:02.463227652 (5h37m16.050283873s) [upload=5775]: Waiting for error group managing segments... Aug 7 22:04:02.463258120 (5h37m16.050314331s) [upload=5775]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:04:02.463261796 (5h37m16.050318008s) [upload=5775]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:04:02.463264733 (5h37m16.050320954s) [upload=5775]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:04:02.530547234 (5h37m16.117603445s) [upload=5775]: Done uploading segments... Aug 7 22:04:02.530562512 (5h37m16.117618733s) [upload=5775]: Upload finished. err: <nil> Aug 7 22:05:10.744065032 (5h38m24.331121272s) [upload=5796]: Starting upload Aug 7 22:05:10.744195716 (5h38m24.331251957s) [upload=5796]: Done starting upload Aug 7 22:05:10.744327762 (5h38m24.331384002s) [upload=5796]: Uploading segments... Aug 7 22:05:10.744369549 (5h38m24.331425791s) [upload=5796]: Got next segment. Inline: true Aug 7 22:05:10.744388655 (5h38m24.331444897s) [upload=5796]: Waiting for error group managing segments... Aug 7 22:05:10.744449028 (5h38m24.331505259s) [upload=5796]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:05:10.744454398 (5h38m24.331510629s) [upload=5796]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:05:10.744457633 (5h38m24.331513875s) [upload=5796]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:05:10.852991578 (5h38m24.440047789s) [upload=5796]: Done uploading segments... Aug 7 22:05:10.853004571 (5h38m24.440060782s) [upload=5796]: Upload finished. err: <nil> Aug 7 22:05:45.060720865 (5h38m58.647777086s) [upload=5806]: Starting upload Aug 7 22:05:45.060780046 (5h38m58.647836267s) [upload=5806]: Done starting upload Aug 7 22:05:45.060842121 (5h38m58.647898333s) [upload=5806]: Uploading segments... Aug 7 22:05:45.060876836 (5h38m58.647933057s) [upload=5806]: Got next segment. Inline: true Aug 7 22:05:45.060882878 (5h38m58.647939099s) [upload=5806]: Waiting for error group managing segments... Aug 7 22:05:45.060927901 (5h38m58.647984112s) [upload=5806]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:05:45.060933843 (5h38m58.647990064s) [upload=5806]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:05:45.060936187 (5h38m58.647992408s) [upload=5806]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:05:45.328399453 (5h38m58.915455674s) [upload=5806]: Done uploading segments... Aug 7 22:05:45.328433367 (5h38m58.915489588s) [upload=5806]: Upload finished. err: <nil> Aug 7 22:06:12.931860875 (5h39m26.518917096s) [upload=5815]: Starting upload Aug 7 22:06:12.931905979 (5h39m26.51896219s) [upload=5815]: Done starting upload Aug 7 22:06:12.931964518 (5h39m26.51902073s) [upload=5815]: Uploading segments... Aug 7 22:06:12.931990617 (5h39m26.519046827s) [upload=5815]: Got next segment. Inline: true Aug 7 22:06:12.931994224 (5h39m26.519050434s) [upload=5815]: Waiting for error group managing segments... Aug 7 22:06:12.932024821 (5h39m26.519081042s) [upload=5815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:06:12.932028408 (5h39m26.519084618s) [upload=5815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:06:12.932030893 (5h39m26.519087103s) [upload=5815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:06:13.014995076 (5h39m26.602051317s) [upload=5815]: Done uploading segments... Aug 7 22:06:13.015014863 (5h39m26.602071114s) [upload=5815]: Upload finished. err: <nil> Aug 7 22:07:16.916734820 (5h40m30.503791062s) [upload=5832]: Starting upload Aug 7 22:07:16.916800323 (5h40m30.503856563s) [upload=5832]: Done starting upload Aug 7 22:07:16.916864883 (5h40m30.503921123s) [upload=5832]: Uploading segments... Aug 7 22:07:16.916903304 (5h40m30.503959546s) [upload=5832]: Got next segment. Inline: true Aug 7 22:07:16.916908815 (5h40m30.503965056s) [upload=5832]: Waiting for error group managing segments... Aug 7 22:07:16.916952356 (5h40m30.504008598s) [upload=5832]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:07:16.916957657 (5h40m30.504013897s) [upload=5832]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:07:16.916961754 (5h40m30.504017995s) [upload=5832]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:07:16.988776316 (5h40m30.575832566s) [upload=5832]: Done uploading segments... Aug 7 22:07:16.988801573 (5h40m30.575857824s) [upload=5832]: Upload finished. err: <nil> Aug 7 22:08:29.650294124 (5h41m43.237350365s) [upload=5854]: Starting upload Aug 7 22:08:29.650357383 (5h41m43.237413623s) [upload=5854]: Done starting upload Aug 7 22:08:29.650418325 (5h41m43.237474567s) [upload=5854]: Uploading segments... Aug 7 22:08:29.650468490 (5h41m43.237524731s) [upload=5854]: Got next segment. Inline: true Aug 7 22:08:29.650474350 (5h41m43.237530592s) [upload=5854]: Waiting for error group managing segments... Aug 7 22:08:29.650515618 (5h41m43.237571859s) [upload=5854]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:08:29.650522029 (5h41m43.237578271s) [upload=5854]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:08:29.650525556 (5h41m43.237581797s) [upload=5854]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:08:29.706715553 (5h41m43.293771793s) [upload=5854]: Done uploading segments... Aug 7 22:08:29.706731001 (5h41m43.293787243s) [upload=5854]: Upload finished. err: metaclient: Too Many Requests Duration:* 238.289289ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 21 out of 60 is failed (threshold=3) [XXXX.X.XX...............................XXXXXXXXXXX.X.XX....] duration: 138.684001ms FAIL (2025-08-07 22:31:12 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 04]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:19:04.377299713 (5h52m17.964355924s) [upload=6018]: Starting upload Aug 7 22:19:04.377350166 (5h52m17.964406387s) [upload=6018]: Done starting upload Aug 7 22:19:04.377401523 (5h52m17.964457744s) [upload=6018]: Uploading segments... Aug 7 22:19:04.377446897 (5h52m17.964503118s) [upload=6018]: Got next segment. Inline: true Aug 7 22:19:04.377453410 (5h52m17.96450962s) [upload=6018]: Waiting for error group managing segments... Aug 7 22:19:04.377487142 (5h52m17.964543354s) [upload=6018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:19:04.377491361 (5h52m17.964547582s) [upload=6018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:19:04.377493875 (5h52m17.964550086s) [upload=6018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:20:04.361964803 (5h53m17.949021024s) [upload=6018]: Done uploading segments... Aug 7 22:20:04.361979240 (5h53m17.949035461s) [upload=6018]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:20:11.847632631 (5h53m25.434688843s) [upload=6035]: Starting upload Aug 7 22:20:11.847694537 (5h53m25.434750747s) [upload=6035]: Done starting upload Aug 7 22:20:11.847740192 (5h53m25.434796403s) [upload=6035]: Uploading segments... Aug 7 22:20:11.847772231 (5h53m25.434828442s) [upload=6035]: Got next segment. Inline: true Aug 7 22:20:11.847776860 (5h53m25.434833071s) [upload=6035]: Waiting for error group managing segments... Aug 7 22:20:11.847824679 (5h53m25.434880891s) [upload=6035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:20:11.847830130 (5h53m25.434886341s) [upload=6035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:20:11.847833205 (5h53m25.434889417s) [upload=6035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:21:11.836000640 (5h54m25.42305688s) [upload=6035]: Done uploading segments... Aug 7 22:21:11.836026087 (5h54m25.423082328s) [upload=6035]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:21:11.965120248 (5h54m25.552176489s) [upload=6050]: Starting upload Aug 7 22:21:11.965231515 (5h54m25.552287757s) [upload=6050]: Done starting upload Aug 7 22:21:11.965301156 (5h54m25.552357397s) [upload=6050]: Uploading segments... Aug 7 22:21:11.965348053 (5h54m25.552404304s) [upload=6050]: Got next segment. Inline: true Aug 7 22:21:11.965353994 (5h54m25.552410235s) [upload=6050]: Waiting for error group managing segments... Aug 7 22:21:11.965399049 (5h54m25.55245529s) [upload=6050]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:21:11.965405170 (5h54m25.552461411s) [upload=6050]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:21:11.965408957 (5h54m25.552465198s) [upload=6050]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:22:11.952333050 (5h55m25.539389261s) [upload=6050]: Done uploading segments... Aug 7 22:22:11.952348108 (5h55m25.539404329s) [upload=6050]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:22:12.125652873 (5h55m25.712709094s) [upload=6063]: Starting upload Aug 7 22:22:12.125702476 (5h55m25.712758688s) [upload=6063]: Done starting upload Aug 7 22:22:12.125747020 (5h55m25.712803241s) [upload=6063]: Uploading segments... Aug 7 22:22:12.125776895 (5h55m25.712833116s) [upload=6063]: Got next segment. Inline: true Aug 7 22:22:12.125781404 (5h55m25.712837625s) [upload=6063]: Waiting for error group managing segments... Aug 7 22:22:12.125814315 (5h55m25.712870536s) [upload=6063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:22:12.125818052 (5h55m25.712874263s) [upload=6063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:22:12.125820397 (5h55m25.712876618s) [upload=6063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:23:12.024254277 (5h56m25.611310498s) [upload=6063]: Done uploading segments... Aug 7 22:23:12.024289072 (5h56m25.611345283s) [upload=6063]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:23:12.175471355 (5h56m25.762527616s) [upload=6082]: Starting upload Aug 7 22:23:12.175573175 (5h56m25.762629466s) [upload=6082]: Done starting upload Aug 7 22:23:12.175670356 (5h56m25.762726627s) [upload=6082]: Uploading segments... Aug 7 22:23:12.175760093 (5h56m25.762816364s) [upload=6082]: Got next segment. Inline: true Aug 7 22:23:12.175767677 (5h56m25.762823919s) [upload=6082]: Waiting for error group managing segments... Aug 7 22:23:12.175819345 (5h56m25.762875585s) [upload=6082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:23:12.175825105 (5h56m25.762881346s) [upload=6082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:23:12.175828732 (5h56m25.762884973s) [upload=6082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:24:12.154559214 (5h57m25.741615455s) [upload=6082]: Done uploading segments... Aug 7 22:24:12.154578851 (5h57m25.741635092s) [upload=6082]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:24:12.253523032 (5h57m25.840579253s) [upload=6098]: Starting upload Aug 7 22:24:12.253572454 (5h57m25.840628665s) [upload=6098]: Done starting upload Aug 7 22:24:12.253615034 (5h57m25.840671245s) [upload=6098]: Uploading segments... Aug 7 22:24:12.253654808 (5h57m25.840711029s) [upload=6098]: Got next segment. Inline: true Aug 7 22:24:12.253658636 (5h57m25.840714847s) [upload=6098]: Waiting for error group managing segments... Aug 7 22:24:12.253689122 (5h57m25.840745343s) [upload=6098]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:24:12.253692969 (5h57m25.84074918s) [upload=6098]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:12.253695303 (5h57m25.840751524s) [upload=6098]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:24:12.467343237 (5h57m26.054399448s) [upload=6098]: Done uploading segments... Aug 7 22:24:12.467365428 (5h57m26.05442164s) [upload=6098]: Upload finished. err: <nil> Aug 7 22:24:33.085790933 (5h57m46.672847184s) [upload=6106]: Starting upload Aug 7 22:24:33.085870281 (5h57m46.672926522s) [upload=6106]: Done starting upload Aug 7 22:24:33.085932668 (5h57m46.67298892s) [upload=6106]: Uploading segments... Aug 7 22:24:33.085994053 (5h57m46.673050293s) [upload=6106]: Got next segment. Inline: true Aug 7 22:24:33.086000064 (5h57m46.673056304s) [upload=6106]: Waiting for error group managing segments... Aug 7 22:24:33.086045709 (5h57m46.67310195s) [upload=6106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:24:33.086051168 (5h57m46.67310741s) [upload=6106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:33.086054985 (5h57m46.673111227s) [upload=6106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:33.072711385 (5h58m46.659767636s) [upload=6106]: Done uploading segments... Aug 7 22:25:33.072753384 (5h58m46.659809634s) [upload=6106]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:25:33.160012555 (5h58m46.747068795s) [upload=6120]: Starting upload Aug 7 22:25:33.160101981 (5h58m46.747158222s) [upload=6120]: Done starting upload Aug 7 22:25:33.160199473 (5h58m46.747255715s) [upload=6120]: Uploading segments... Aug 7 22:25:33.160243596 (5h58m46.747299837s) [upload=6120]: Got next segment. Inline: true Aug 7 22:25:33.160249046 (5h58m46.747305287s) [upload=6120]: Waiting for error group managing segments... Aug 7 22:25:33.160294231 (5h58m46.747350482s) [upload=6120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:25:33.160300012 (5h58m46.747356252s) [upload=6120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:25:33.160303619 (5h58m46.747359859s) [upload=6120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:33.228188865 (5h58m46.815245086s) [upload=6120]: Done uploading segments... Aug 7 22:25:33.228202571 (5h58m46.815258782s) [upload=6120]: Upload finished. err: <nil> Aug 7 22:26:14.972993755 (5h59m28.560049967s) [upload=6132]: Starting upload Aug 7 22:26:14.973047095 (5h59m28.560103307s) [upload=6132]: Done starting upload Aug 7 22:26:14.973101456 (5h59m28.560157677s) [upload=6132]: Uploading segments... Aug 7 22:26:14.973137704 (5h59m28.560193925s) [upload=6132]: Got next segment. Inline: true Aug 7 22:26:14.973141382 (5h59m28.560197592s) [upload=6132]: Waiting for error group managing segments... Aug 7 22:26:14.973187948 (5h59m28.560244158s) [upload=6132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:26:14.973191995 (5h59m28.560248217s) [upload=6132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:26:14.973195201 (5h59m28.560251423s) [upload=6132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:14.950324988 (6h0m28.537381229s) [upload=6132]: Done uploading segments... Aug 7 22:27:14.950345787 (6h0m28.537402028s) [upload=6132]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:27:22.925128081 (6h0m36.512184292s) [upload=6146]: Starting upload Aug 7 22:27:22.925194244 (6h0m36.512250465s) [upload=6146]: Done starting upload Aug 7 22:27:22.925235571 (6h0m36.512291783s) [upload=6146]: Uploading segments... Aug 7 22:27:22.925260508 (6h0m36.512316729s) [upload=6146]: Got next segment. Inline: true Aug 7 22:27:22.925264486 (6h0m36.512320707s) [upload=6146]: Waiting for error group managing segments... Aug 7 22:27:22.925305752 (6h0m36.512361973s) [upload=6146]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:22.925309750 (6h0m36.512365971s) [upload=6146]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:22.925312044 (6h0m36.512368265s) [upload=6146]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:28:22.913115749 (6h1m36.50017199s) [upload=6146]: Done uploading segments... Aug 7 22:28:22.913136929 (6h1m36.500193169s) [upload=6146]: Upload finished. err: metaclient: context deadline exceeded Duration: 138.707466ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 169.758066ms remote: "tcp:136.0.77.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 22:31:10 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 169.783082ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 21 out of 60 is failed (threshold=3) [XXXXX.XX......X.............................XXXXXXXXXXXXX...] duration: 612.462662ms FAIL (2025-08-07 22:31:08 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== [upload=6089]: Done starting upload Aug 7 22:23:43.961837605 (5h56m57.548893826s) [upload=6089]: Uploading segments... Aug 7 22:23:43.962138106 (5h56m57.549194347s) [upload=6089]: Got next segment. Inline: false Aug 7 22:23:43.962204581 (5h56m57.549260822s) [upload=6089]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:23:43.962209980 (5h56m57.549266222s) [upload=6089]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:24:12.375269699 (5h57m25.96232592s) [seg_pos={0 0}, upload=6089]: Begin upload segment... Aug 7 22:24:12.375313250 (5h57m25.962369471s) [seg_pos={0 0}, upload=6089]: Done begin upload segment. Aug 7 22:24:12.375339780 (5h57m25.962396031s) [noise=true , address=193.149.164.143:4698 , piece_id=AV2DCKYWWT5N4PL57WTXW45JNFYOCRLLEY6YSBSWOAQAPHAVXJQQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.375356050 (5h57m25.962412261s) [noise=true , address=104.36.82.53:4156 , piece_id=NBWZ4FB4S7CTDN2OTWTDKSH2R7DJN66P7GRTQTQ5FJZFQCZLDAQQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.375715481 (5h57m25.962771693s) [noise=true , address=104.36.82.58:4280 , piece_id=W2A65H4FIUOCUSMSVCANR2OPVBUFO4YNAFZV3KCASSCSPCHONGBA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.375724769 (5h57m25.96278099s) [noise=true , address=104.36.82.59:4333 , piece_id=JONL6XDWEUYMHVDPJVB54PE5HCYWN7KRO3FSNAMPJWCEP6QBKYVQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376022925 (5h57m25.963079146s) [noise=true , address=45.91.116.9:4270 , piece_id=CMJ7K7XDV44UUOWXU6NOUNMWZGLOEZKPTLVFMTNWIR6OK52TOEKQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376075263 (5h57m25.963131475s) [noise=true , address=104.36.82.36:4652 , piece_id=ZDJ2LSAX25MBAAW4PW745SX7J7X3S4PAGXEUS36T43KTM3LZ2WRQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376490167 (5h57m25.963546388s) [noise=true , address=104.36.82.45:4921 , piece_id=DCEBY7YTNAO4UR6YZ5LC5W27INFXZQ26BPJ7BHPMXY26P5BDL23Q, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376527327 (5h57m25.963583599s) [noise=true , address=104.36.82.12:4169 , piece_id=XPNXIWBPTKCEEQTVQ3BAQ3HCLQE5U5BTRIAR3FQ6O4OQJZMCJ6ZA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376755913 (5h57m25.963812125s) [noise=true , address=104.36.82.62:4411 , piece_id=3TMITFZMXF5ESOCJNLFU6EIT7CFUYULYZ6FJUQ6X7T2WBEQMKWFQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376864006 (5h57m25.963920216s) [noise=true , address=104.36.82.14:4243 , piece_id=WKT5D56ZXVSJJHVR3GB6N5XR6CGU4VW2XK2YTMPBZOH4S2ZPOVCA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376979641 (5h57m25.964035862s) [noise=true , address=104.36.82.9:4031 , piece_id=UOVVYSA77DQOK53ATDDXV56PLEVS6GBQ5HMZEKCEURYBPRCXNYPA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.377222374 (5h57m25.964278595s) [noise=true , address=104.36.82.52:4100 , piece_id=7RUPILCVG3GL3KB4CS25YCYKDB4C6HFT6TGZWZIDC5EHBI2SD55Q, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.377246499 (5h57m25.96430271s) [noise=true , address=79.127.201.215:4343 , piece_id=N3KGRRGXWJLCJGYSLPOLOIYZHP7AFWV6VBCHVBETDYOQBIXIWPMQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.377620267 (5h57m25.964676488s) [noise=true , address=104.36.82.35:4597 , piece_id=6UMSPJQOVLEDWMIQMLAJLZ2AY6Y5XLMODCN3V63EB5JNJS3EC7BA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.377642138 (5h57m25.964698349s) [noise=true , address=104.36.82.16:4300 , piece_id=RJRU7SETYXWMQK7GEE3HBD2XMEVSI3UJ6ZDCIWVW4266QSNAOPZA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.377932059 (5h57m25.96498828s) [noise=true , address=104.36.82.47:4544 , piece_id=DVQRASYHRKPFEONZEWJ3MBVM7O5764NPWYADLMG4ARAH5PBD55MA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.377950764 (5h57m25.965006985s) [noise=true , address=104.36.82.21:4478 , piece_id=5UO756WUOKCS52UJI2NIQFUSU7W6NBW34MGFH6YZX27HKWY5243A, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.378280479 (5h57m25.9653367s) [noise=true , address=104.36.82.37:4656 , piece_id=A25QOY4UFT7RRRY6PVAS3NNML7PGJM4OHDH56PF2HZHKJ7VOPURA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.468526485 (5h57m26.055582696s) [upload=6089]: Next returned nil segment Aug 7 22:24:12.468535081 (5h57m26.055591292s) [upload=6089]: Waiting for error group managing segments... Aug 7 22:24:12.483383738 (5h57m26.070439949s) [noise=true , address=104.36.82.37:4656 , piece_id=A25QOY4UFT7RRRY6PVAS3NNML7PGJM4OHDH56PF2HZHKJ7VOPURA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.500821799 (5h57m26.08787801s) [noise=true , address=104.36.82.16:4300 , piece_id=RJRU7SETYXWMQK7GEE3HBD2XMEVSI3UJ6ZDCIWVW4266QSNAOPZA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.504410257 (5h57m26.091466458s) [noise=true , address=45.91.116.9:4270 , piece_id=CMJ7K7XDV44UUOWXU6NOUNMWZGLOEZKPTLVFMTNWIR6OK52TOEKQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.511708921 (5h57m26.098765132s) [noise=true , address=104.36.82.35:4597 , piece_id=6UMSPJQOVLEDWMIQMLAJLZ2AY6Y5XLMODCN3V63EB5JNJS3EC7BA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.523758022 (5h57m26.110814233s) [noise=true , address=104.36.82.12:4169 , piece_id=XPNXIWBPTKCEEQTVQ3BAQ3HCLQE5U5BTRIAR3FQ6O4OQJZMCJ6ZA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.533614250 (5h57m26.120670461s) [noise=true , address=104.36.82.52:4100 , piece_id=7RUPILCVG3GL3KB4CS25YCYKDB4C6HFT6TGZWZIDC5EHBI2SD55Q, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.557032764 (5h57m26.144088985s) [noise=true , address=104.36.82.21:4478 , piece_id=5UO756WUOKCS52UJI2NIQFUSU7W6NBW34MGFH6YZX27HKWY5243A, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.582698011 (5h57m26.169754223s) [noise=true , address=104.36.82.53:4156 , piece_id=NBWZ4FB4S7CTDN2OTWTDKSH2R7DJN66P7GRTQTQ5FJZFQCZLDAQQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.609343038 (5h57m26.196399279s) [noise=true , address=79.127.201.215:4343 , piece_id=N3KGRRGXWJLCJGYSLPOLOIYZHP7AFWV6VBCHVBETDYOQBIXIWPMQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.639134328 (5h57m26.226190538s) [noise=true , address=104.36.82.36:4652 , piece_id=ZDJ2LSAX25MBAAW4PW745SX7J7X3S4PAGXEUS36T43KTM3LZ2WRQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.956879031 (5h57m26.543935272s) [noise=true , address=193.149.164.143:4698 , piece_id=AV2DCKYWWT5N4PL57WTXW45JNFYOCRLLEY6YSBSWOAQAPHAVXJQQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:13.036416979 (5h57m26.623473231s) [noise=true , address=104.36.82.9:4031 , piece_id=UOVVYSA77DQOK53ATDDXV56PLEVS6GBQ5HMZEKCEURYBPRCXNYPA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:13.172901316 (5h57m26.759957557s) [noise=true , address=104.36.82.47:4544 , piece_id=DVQRASYHRKPFEONZEWJ3MBVM7O5764NPWYADLMG4ARAH5PBD55MA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:13.202754351 (5h57m26.789810592s) [noise=true , address=104.36.82.14:4243 , piece_id=WKT5D56ZXVSJJHVR3GB6N5XR6CGU4VW2XK2YTMPBZOH4S2ZPOVCA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:13.202797742 (5h57m26.789853984s) [seg_pos={0 0}, upload=6089]: Segment reached optimal threshold of 14 pieces. Aug 7 22:24:13.203007333 (5h57m26.790063585s) [noise=true , address=104.36.82.58:4280 , piece_id=W2A65H4FIUOCUSMSVCANR2OPVBUFO4YNAFZV3KCASSCSPCHONGBA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fgHaEFKK4CeGg4x3vywTuYfT3kmac31xqg2QpsahQNXmvjhW8): protocol: expected piece hash; context canceled; EOF Aug 7 22:24:13.203190164 (5h57m26.790246415s) [noise=true , address=104.36.82.59:4333 , piece_id=JONL6XDWEUYMHVDPJVB54PE5HCYWN7KRO3FSNAMPJWCEP6QBKYVQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K7U6NXcB6SFBtQFrG7DXt2skACr3u5DX2QsrnToi8EKywS9TN): protocol: write tcp 5.161.89.252:63730->104.36.82.59:4333: use of closed network connection; write tcp 5.161.89.252:63730->104.36.82.59:4333: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:63730->104.36.82.59:4333: use of closed network connection Aug 7 22:24:13.203221153 (5h57m26.790277394s) [noise=true , address=104.36.82.45:4921 , piece_id=DCEBY7YTNAO4UR6YZ5LC5W27INFXZQ26BPJ7BHPMXY26P5BDL23Q, seg_pos={0 0}, upload=6089]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bBRKXiuSN2cw6KrdP8q4Dsf4MiFCUSgMEGjqG2dKL9RLTQra1): protocol: expected piece hash; context canceled; EOF Aug 7 22:24:13.203424783 (5h57m26.790481023s) [noise=true , address=104.36.82.62:4411 , piece_id=3TMITFZMXF5ESOCJNLFU6EIT7CFUYULYZ6FJUQ6X7T2WBEQMKWFQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DT68m6J4PpDk3XUDhDaaqV46NLKVz4A23716JLFxEAh122GeR): protocol: context canceled; context canceled Aug 7 22:24:13.203452675 (5h57m26.790508916s) [seg_pos={0 0}, upload=6089]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 22:24:13.203591695 (5h57m26.790647935s) [upload=6089]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:24:13.203602313 (5h57m26.790658565s) [upload=6089]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:29:03.852878645 (6h2m17.439934866s) [upload=6089]: Done uploading segments... Aug 7 22:29:03.852904053 (6h2m17.439960294s) [upload=6089]: Upload finished. err: metaclient: context deadline exceeded Duration: 612.486687ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 27 out of 60 is failed (threshold=3) [XXXXX.............................XXXXXXXXXXXXXXXX.XXXXX.X..] duration: 172.092805ms FAIL (2025-08-07 22:31:05 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 420501s) [upload=6055]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:22:06.462569787 (5h55m20.049626008s) [upload=6061]: Starting upload Aug 7 22:22:06.462655577 (5h55m20.049711788s) [upload=6061]: Done starting upload Aug 7 22:22:06.462750936 (5h55m20.049807157s) [upload=6061]: Uploading segments... Aug 7 22:22:06.462782464 (5h55m20.049838675s) [upload=6061]: Got next segment. Inline: true Aug 7 22:22:06.462786181 (5h55m20.049842402s) [upload=6061]: Waiting for error group managing segments... Aug 7 22:22:06.462817791 (5h55m20.049874001s) [upload=6061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:22:06.462821648 (5h55m20.049877869s) [upload=6061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:22:06.462823982 (5h55m20.049880203s) [upload=6061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:22:36.449657257 (5h55m50.036713498s) [upload=6061]: Done uploading segments... Aug 7 22:22:36.449678196 (5h55m50.036734448s) [upload=6061]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:22:59.911739779 (5h56m13.498795989s) [upload=6078]: Starting upload Aug 7 22:22:59.911786736 (5h56m13.498842957s) [upload=6078]: Done starting upload Aug 7 22:22:59.911827472 (5h56m13.498883683s) [upload=6078]: Uploading segments... Aug 7 22:22:59.911861074 (5h56m13.498917295s) [upload=6078]: Got next segment. Inline: true Aug 7 22:22:59.911864401 (5h56m13.498920622s) [upload=6078]: Waiting for error group managing segments... Aug 7 22:22:59.911889107 (5h56m13.498945319s) [upload=6078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:22:59.911893005 (5h56m13.498949226s) [upload=6078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:22:59.911894898 (5h56m13.498951119s) [upload=6078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:23:29.900458747 (5h56m43.487514958s) [upload=6078]: Done uploading segments... Aug 7 22:23:29.900473835 (5h56m43.487530047s) [upload=6078]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:23:58.370463254 (5h57m11.957519465s) [upload=6094]: Starting upload Aug 7 22:23:58.370512907 (5h57m11.957569128s) [upload=6094]: Done starting upload Aug 7 22:23:58.370553031 (5h57m11.957609243s) [upload=6094]: Uploading segments... Aug 7 22:23:58.370578639 (5h57m11.95763486s) [upload=6094]: Got next segment. Inline: true Aug 7 22:23:58.370582427 (5h57m11.957638648s) [upload=6094]: Waiting for error group managing segments... Aug 7 22:23:58.370610951 (5h57m11.957667172s) [upload=6094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:23:58.370615178 (5h57m11.957671399s) [upload=6094]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:23:58.370617392 (5h57m11.957673613s) [upload=6094]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:23:58.450972793 (5h57m12.038029004s) [upload=6094]: Done uploading segments... Aug 7 22:23:58.450986078 (5h57m12.038042289s) [upload=6094]: Upload finished. err: <nil> Aug 7 22:24:40.794473113 (5h57m54.381529325s) [upload=6107]: Starting upload Aug 7 22:24:40.794523428 (5h57m54.381579639s) [upload=6107]: Done starting upload Aug 7 22:24:40.794565206 (5h57m54.381621417s) [upload=6107]: Uploading segments... Aug 7 22:24:40.794591815 (5h57m54.381648026s) [upload=6107]: Got next segment. Inline: true Aug 7 22:24:40.794595622 (5h57m54.381651833s) [upload=6107]: Waiting for error group managing segments... Aug 7 22:24:40.794624367 (5h57m54.381680577s) [upload=6107]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:24:40.794628323 (5h57m54.381684535s) [upload=6107]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:40.794630638 (5h57m54.381686849s) [upload=6107]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:10.774549311 (5h58m24.361605533s) [upload=6107]: Done uploading segments... Aug 7 22:25:10.774570019 (5h58m24.361626231s) [upload=6107]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:25:10.855666099 (5h58m24.442722339s) [upload=6114]: Starting upload Aug 7 22:25:10.855752680 (5h58m24.442808921s) [upload=6114]: Done starting upload Aug 7 22:25:10.855822740 (5h58m24.442878991s) [upload=6114]: Uploading segments... Aug 7 22:25:10.855866683 (5h58m24.442922923s) [upload=6114]: Got next segment. Inline: true Aug 7 22:25:10.855872564 (5h58m24.442928805s) [upload=6114]: Waiting for error group managing segments... Aug 7 22:25:10.855966008 (5h58m24.443022249s) [upload=6114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:25:10.855972971 (5h58m24.443029212s) [upload=6114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:25:10.855976568 (5h58m24.443032819s) [upload=6114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:40.842457902 (5h58m54.429514153s) [upload=6114]: Done uploading segments... Aug 7 22:25:40.842479021 (5h58m54.429535272s) [upload=6114]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:25:40.940807525 (5h58m54.527863766s) [upload=6122]: Starting upload Aug 7 22:25:40.940896992 (5h58m54.527953233s) [upload=6122]: Done starting upload Aug 7 22:25:40.940964388 (5h58m54.528020628s) [upload=6122]: Uploading segments... Aug 7 22:25:40.941010434 (5h58m54.528066675s) [upload=6122]: Got next segment. Inline: true Aug 7 22:25:40.941016375 (5h58m54.528072616s) [upload=6122]: Waiting for error group managing segments... Aug 7 22:25:40.941060899 (5h58m54.52811715s) [upload=6122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:25:40.941068091 (5h58m54.528124332s) [upload=6122]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:25:40.941073060 (5h58m54.528129311s) [upload=6122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:26:10.929528068 (5h59m24.516584319s) [upload=6122]: Done uploading segments... Aug 7 22:26:10.929548557 (5h59m24.516604797s) [upload=6122]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:26:11.008231659 (5h59m24.59528788s) [upload=6129]: Starting upload Aug 7 22:26:11.008286501 (5h59m24.595342713s) [upload=6129]: Done starting upload Aug 7 22:26:11.008333198 (5h59m24.595389419s) [upload=6129]: Uploading segments... Aug 7 22:26:11.008364838 (5h59m24.595421059s) [upload=6129]: Got next segment. Inline: true Aug 7 22:26:11.008368595 (5h59m24.595424816s) [upload=6129]: Waiting for error group managing segments... Aug 7 22:26:11.008399943 (5h59m24.595456164s) [upload=6129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:26:11.008403599 (5h59m24.59545981s) [upload=6129]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:26:11.008406565 (5h59m24.595462786s) [upload=6129]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:26:40.996305184 (5h59m54.583361385s) [upload=6129]: Done uploading segments... Aug 7 22:26:40.996323769 (5h59m54.58337998s) [upload=6129]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:27:14.829073707 (6h0m28.416129948s) [upload=6144]: Starting upload Aug 7 22:27:14.829153145 (6h0m28.416209386s) [upload=6144]: Done starting upload Aug 7 22:27:14.829247050 (6h0m28.416303291s) [upload=6144]: Uploading segments... Aug 7 22:27:14.829299899 (6h0m28.41635614s) [upload=6144]: Got next segment. Inline: true Aug 7 22:27:14.829306702 (6h0m28.416362943s) [upload=6144]: Waiting for error group managing segments... Aug 7 22:27:14.829350864 (6h0m28.416407105s) [upload=6144]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:14.829357416 (6h0m28.416413657s) [upload=6144]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:14.829361143 (6h0m28.416417384s) [upload=6144]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:44.808549878 (6h0m58.395606099s) [upload=6144]: Done uploading segments... Aug 7 22:27:44.808564877 (6h0m58.395621098s) [upload=6144]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:28:23.619901331 (6h1m37.206957573s) [upload=6162]: Starting upload Aug 7 22:28:23.620017008 (6h1m37.207073248s) [upload=6162]: Done starting upload Aug 7 22:28:23.620072551 (6h1m37.207128791s) [upload=6162]: Uploading segments... Aug 7 22:28:23.620125059 (6h1m37.2071813s) [upload=6162]: Got next segment. Inline: true Aug 7 22:28:23.620132432 (6h1m37.207188672s) [upload=6162]: Waiting for error group managing segments... Aug 7 22:28:23.620198335 (6h1m37.207254577s) [upload=6162]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:28:23.620204407 (6h1m37.207260647s) [upload=6162]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:28:23.620208153 (6h1m37.207264395s) [upload=6162]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:28:23.683041871 (6h1m37.270098112s) [upload=6162]: Done uploading segments... Aug 7 22:28:23.683055537 (6h1m37.270111778s) [upload=6162]: Upload finished. err: <nil> Aug 7 22:28:40.804037302 (6h1m54.391093523s) [upload=6165]: Starting upload Aug 7 22:28:40.804089460 (6h1m54.391145671s) [upload=6165]: Done starting upload Aug 7 22:28:40.804134294 (6h1m54.391190515s) [upload=6165]: Uploading segments... Aug 7 22:28:40.804181422 (6h1m54.391237632s) [upload=6165]: Got next segment. Inline: true Aug 7 22:28:40.804185379 (6h1m54.3912416s) [upload=6165]: Waiting for error group managing segments... Aug 7 22:28:40.804217198 (6h1m54.391273419s) [upload=6165]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:28:40.804220915 (6h1m54.391277127s) [upload=6165]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:28:40.804223370 (6h1m54.391279591s) [upload=6165]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:29:10.793781791 (6h2m24.380838023s) [upload=6165]: Done uploading segments... Aug 7 22:29:10.793803392 (6h2m24.380859603s) [upload=6165]: Upload finished. err: metaclient: context deadline exceeded Duration: 172.1167ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 21 out of 60 is failed (threshold=3) [.....X..............................XXXXXXXXXXXXXXXXX..XX..X] duration: 30.000801015s FAIL (2025-08-07 22:30:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000819539s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 1.582219156s FAIL (2025-08-07 22:30:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ine Aug 7 21:59:56.515336538 (5h33m10.10239275s) [upload=5708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:59:56.574152557 (5h33m10.161208778s) [upload=5708]: Done uploading segments... Aug 7 21:59:56.574171943 (5h33m10.161228164s) [upload=5708]: Upload finished. err: <nil> Aug 7 22:00:36.611785100 (5h33m50.19884134s) [upload=5716]: Starting upload Aug 7 22:00:36.611856203 (5h33m50.198912444s) [upload=5716]: Done starting upload Aug 7 22:00:36.611959786 (5h33m50.199016027s) [upload=5716]: Uploading segments... Aug 7 22:00:36.612003668 (5h33m50.199059918s) [upload=5716]: Got next segment. Inline: true Aug 7 22:00:36.612009668 (5h33m50.19906591s) [upload=5716]: Waiting for error group managing segments... Aug 7 22:00:36.612057077 (5h33m50.199113319s) [upload=5716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:00:36.612082536 (5h33m50.199138776s) [upload=5716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:00:36.612086222 (5h33m50.199142473s) [upload=5716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:00:36.884781084 (5h33m50.471837294s) [upload=5716]: Done uploading segments... Aug 7 22:00:36.884795611 (5h33m50.471851832s) [upload=5716]: Upload finished. err: <nil> Aug 7 22:01:47.555449092 (5h35m1.142505313s) [upload=5736]: Starting upload Aug 7 22:01:47.555491781 (5h35m1.142548002s) [upload=5736]: Done starting upload Aug 7 22:01:47.555562123 (5h35m1.142618344s) [upload=5736]: Uploading segments... Aug 7 22:01:47.555595826 (5h35m1.142652047s) [upload=5736]: Got next segment. Inline: true Aug 7 22:01:47.555599363 (5h35m1.142655584s) [upload=5736]: Waiting for error group managing segments... Aug 7 22:01:47.555631602 (5h35m1.142687823s) [upload=5736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:47.555635390 (5h35m1.142691611s) [upload=5736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:47.555637894 (5h35m1.142694115s) [upload=5736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:47.686110188 (5h35m1.273166429s) [upload=5736]: Done uploading segments... Aug 7 22:01:47.686151165 (5h35m1.273207406s) [upload=5736]: Upload finished. err: <nil> Aug 7 22:02:22.678566055 (5h35m36.265622267s) [upload=5746]: Starting upload Aug 7 22:02:22.678633091 (5h35m36.265689303s) [upload=5746]: Done starting upload Aug 7 22:02:22.678684407 (5h35m36.265740617s) [upload=5746]: Uploading segments... Aug 7 22:02:22.678721757 (5h35m36.265777967s) [upload=5746]: Got next segment. Inline: true Aug 7 22:02:22.678725724 (5h35m36.265781935s) [upload=5746]: Waiting for error group managing segments... Aug 7 22:02:22.678759457 (5h35m36.265815668s) [upload=5746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:02:22.678764947 (5h35m36.265821168s) [upload=5746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:02:22.678768565 (5h35m36.265824826s) [upload=5746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:02:22.971349697 (5h35m36.558405918s) [upload=5746]: Done uploading segments... Aug 7 22:02:22.971373742 (5h35m36.558429963s) [upload=5746]: Upload finished. err: <nil> Aug 7 22:03:08.907032448 (5h36m22.494088689s) [upload=5757]: Starting upload Aug 7 22:03:08.907091528 (5h36m22.494147769s) [upload=5757]: Done starting upload Aug 7 22:03:08.907172368 (5h36m22.4942286s) [upload=5757]: Uploading segments... Aug 7 22:03:08.907209458 (5h36m22.494265699s) [upload=5757]: Got next segment. Inline: true Aug 7 22:03:08.907214758 (5h36m22.494271s) [upload=5757]: Waiting for error group managing segments... Aug 7 22:03:08.907255936 (5h36m22.494312176s) [upload=5757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:03:08.907261175 (5h36m22.494317416s) [upload=5757]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:03:08.907265904 (5h36m22.494322135s) [upload=5757]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:03:09.004207779 (5h36m22.59126404s) [upload=5757]: Done uploading segments... Aug 7 22:03:09.004232375 (5h36m22.591288627s) [upload=5757]: Upload finished. err: <nil> Aug 7 22:04:02.463086309 (5h37m16.050142521s) [upload=5775]: Starting upload Aug 7 22:04:02.463128347 (5h37m16.050184568s) [upload=5775]: Done starting upload Aug 7 22:04:02.463197818 (5h37m16.050254029s) [upload=5775]: Uploading segments... Aug 7 22:04:02.463223595 (5h37m16.050279816s) [upload=5775]: Got next segment. Inline: true Aug 7 22:04:02.463227652 (5h37m16.050283873s) [upload=5775]: Waiting for error group managing segments... Aug 7 22:04:02.463258120 (5h37m16.050314331s) [upload=5775]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:04:02.463261796 (5h37m16.050318008s) [upload=5775]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:04:02.463264733 (5h37m16.050320954s) [upload=5775]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:04:02.530547234 (5h37m16.117603445s) [upload=5775]: Done uploading segments... Aug 7 22:04:02.530562512 (5h37m16.117618733s) [upload=5775]: Upload finished. err: <nil> Aug 7 22:05:10.744065032 (5h38m24.331121272s) [upload=5796]: Starting upload Aug 7 22:05:10.744195716 (5h38m24.331251957s) [upload=5796]: Done starting upload Aug 7 22:05:10.744327762 (5h38m24.331384002s) [upload=5796]: Uploading segments... Aug 7 22:05:10.744369549 (5h38m24.331425791s) [upload=5796]: Got next segment. Inline: true Aug 7 22:05:10.744388655 (5h38m24.331444897s) [upload=5796]: Waiting for error group managing segments... Aug 7 22:05:10.744449028 (5h38m24.331505259s) [upload=5796]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:05:10.744454398 (5h38m24.331510629s) [upload=5796]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:05:10.744457633 (5h38m24.331513875s) [upload=5796]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:05:10.852991578 (5h38m24.440047789s) [upload=5796]: Done uploading segments... Aug 7 22:05:10.853004571 (5h38m24.440060782s) [upload=5796]: Upload finished. err: <nil> Aug 7 22:05:45.060720865 (5h38m58.647777086s) [upload=5806]: Starting upload Aug 7 22:05:45.060780046 (5h38m58.647836267s) [upload=5806]: Done starting upload Aug 7 22:05:45.060842121 (5h38m58.647898333s) [upload=5806]: Uploading segments... Aug 7 22:05:45.060876836 (5h38m58.647933057s) [upload=5806]: Got next segment. Inline: true Aug 7 22:05:45.060882878 (5h38m58.647939099s) [upload=5806]: Waiting for error group managing segments... Aug 7 22:05:45.060927901 (5h38m58.647984112s) [upload=5806]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:05:45.060933843 (5h38m58.647990064s) [upload=5806]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:05:45.060936187 (5h38m58.647992408s) [upload=5806]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:05:45.328399453 (5h38m58.915455674s) [upload=5806]: Done uploading segments... Aug 7 22:05:45.328433367 (5h38m58.915489588s) [upload=5806]: Upload finished. err: <nil> Aug 7 22:06:12.931860875 (5h39m26.518917096s) [upload=5815]: Starting upload Aug 7 22:06:12.931905979 (5h39m26.51896219s) [upload=5815]: Done starting upload Aug 7 22:06:12.931964518 (5h39m26.51902073s) [upload=5815]: Uploading segments... Aug 7 22:06:12.931990617 (5h39m26.519046827s) [upload=5815]: Got next segment. Inline: true Aug 7 22:06:12.931994224 (5h39m26.519050434s) [upload=5815]: Waiting for error group managing segments... Aug 7 22:06:12.932024821 (5h39m26.519081042s) [upload=5815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:06:12.932028408 (5h39m26.519084618s) [upload=5815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:06:12.932030893 (5h39m26.519087103s) [upload=5815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:06:13.014995076 (5h39m26.602051317s) [upload=5815]: Done uploading segments... Aug 7 22:06:13.015014863 (5h39m26.602071114s) [upload=5815]: Upload finished. err: <nil> Aug 7 22:07:16.916734820 (5h40m30.503791062s) [upload=5832]: Starting upload Aug 7 22:07:16.916800323 (5h40m30.503856563s) [upload=5832]: Done starting upload Aug 7 22:07:16.916864883 (5h40m30.503921123s) [upload=5832]: Uploading segments... Aug 7 22:07:16.916903304 (5h40m30.503959546s) [upload=5832]: Got next segment. Inline: true Aug 7 22:07:16.916908815 (5h40m30.503965056s) [upload=5832]: Waiting for error group managing segments... Aug 7 22:07:16.916952356 (5h40m30.504008598s) [upload=5832]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:07:16.916957657 (5h40m30.504013897s) [upload=5832]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:07:16.916961754 (5h40m30.504017995s) [upload=5832]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:07:16.988776316 (5h40m30.575832566s) [upload=5832]: Done uploading segments... Aug 7 22:07:16.988801573 (5h40m30.575857824s) [upload=5832]: Upload finished. err: <nil> Aug 7 22:08:29.650294124 (5h41m43.237350365s) [upload=5854]: Starting upload Aug 7 22:08:29.650357383 (5h41m43.237413623s) [upload=5854]: Done starting upload Aug 7 22:08:29.650418325 (5h41m43.237474567s) [upload=5854]: Uploading segments... Aug 7 22:08:29.650468490 (5h41m43.237524731s) [upload=5854]: Got next segment. Inline: true Aug 7 22:08:29.650474350 (5h41m43.237530592s) [upload=5854]: Waiting for error group managing segments... Aug 7 22:08:29.650515618 (5h41m43.237571859s) [upload=5854]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:08:29.650522029 (5h41m43.237578271s) [upload=5854]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:08:29.650525556 (5h41m43.237581797s) [upload=5854]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:08:29.706715553 (5h41m43.293771793s) [upload=5854]: Done uploading segments... Aug 7 22:08:29.706731001 (5h41m43.293787243s) [upload=5854]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.582239154s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 22 out of 60 is failed (threshold=3) [XXXXXXXXXXXXX..................................XXXXXXXX.X...] duration: 1.685179754s FAIL (2025-08-07 22:30:44 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.685197768s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 21 out of 60 is failed (threshold=3) [..................................XXXXXXXXXXXXXXXX.X.XXXX...] duration: 218.970629ms FAIL (2025-08-07 22:30:41 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 218.994744ms core uplink us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 6 out of 60 is failed (threshold=3) [..................................................XX.XXXX...] duration: 23.340067ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 22:30:41 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 23.362669ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 21 out of 60 is failed (threshold=3) [XXXX....................................XXXXXXXXXXXXXXXX..X.] duration: 1.694488731s FAIL (2025-08-07 22:30:41 UTC) operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.694502718s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 22 out of 60 is failed (threshold=3) [XXXXX....................................XXXXXXXXXXXXXXXX..X] duration: 1m0.001370888s FAIL (2025-08-07 22:30:40 UTC) operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001386638s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [..................................XXXXXXXXXXXXXXXXXXX..XXXX.] duration: 10.178198867s FAIL (2025-08-07 22:30:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 10.178211691s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [....................................XXXXXXXXXXXXXX.XXX......] duration: 601.867527ms FAIL (2025-08-07 22:30:35 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== TVRWLFHTOF2YZFDYX3LU5FENJAS6JOW572GLJ2OHVQ, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.749054955 (5h56m13.336111176s) [noise=true , address=104.36.82.2:4730 , piece_id=FKFZNBUC7L2WVTTFPNRSPJUWW6HAEE6CNUAYSHVDYXW5XENS462Q, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.749426248 (5h56m13.336482459s) [noise=true , address=45.91.116.9:4262 , piece_id=R7J2NVZJTXR7ATH2VUC75B2Z73TSJAQUBYJ3WE75RQ6AD3L3PSCQ, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.750681361 (5h56m13.337737582s) [noise=true , address=104.36.82.5:4840 , piece_id=6SYENOMYMIEYCEFSHJKMXOQE4RCDPDWBAHYDVRTXCQYWJGQDCCKA, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.751093369 (5h56m13.338149581s) [noise=true , address=104.36.82.14:4221 , piece_id=SBKEVRBOXXBFOVTZWPSDVDPXEDV6HZIVAV4MWFIYDMNEU643KBWA, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.751643326 (5h56m13.338699547s) [noise=true , address=45.91.116.4:4940 , piece_id=UXKWVJQC6KYUHGXKHRYHJATOP5KQF4BY2LWNQJQEGFYKQEBT6GWA, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.752249137 (5h56m13.339305348s) [noise=true , address=77.90.31.13:4628 , piece_id=BYQWTXNUD76XIWZZAUZNHDUX7F7CABMDCPK7M7QPDHS3CFBRBMVQ, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.752826465 (5h56m13.339882676s) [noise=true , address=104.36.82.2:4731 , piece_id=L6W56WDLHKGHEPXBK7F2OYZ6CRM66ZVKG5RGJPH3FHC7RZZ3QGBA, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.753473372 (5h56m13.340529583s) [noise=true , address=45.91.116.12:4227 , piece_id=7RIB2VCODFSHTRTBGI3Z74IECO3ATQXSXW7CY3TCA43MZKSKEQYA, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.753961884 (5h56m13.341018096s) [noise=true , address=77.90.31.10:4519 , piece_id=QF2SVPCWB5HG4S7GQL6NFHTYPVOHP4HHORRY2TMJSRXP5A3J43TQ, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.754616727 (5h56m13.341672948s) [noise=true , address=104.36.82.9:4006 , piece_id=MDXFW3M6TBQJXYHMBTPIOD5I5PK3NUHDRZ3EERNCRKFVMUPSW43Q, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.755122451 (5h56m13.342178672s) [noise=true , address=104.36.82.8:4971 , piece_id=2MZZBGEPYJSHOBAG4FUAU5NKKAOL72C2AWD3GZECQXGPMOG6TDMQ, seg_pos={0 0}, upload=6077]: Uploading piece... Aug 7 22:22:59.837536198 (5h56m13.424592419s) [upload=6077]: Next returned nil segment Aug 7 22:22:59.837547418 (5h56m13.424603629s) [upload=6077]: Waiting for error group managing segments... Aug 7 22:22:59.846810959 (5h56m13.43386718s) [noise=true , address=45.91.116.7:4011 , piece_id=E7YGVXQXIO2A34B22AHGKBQXER4KLEX3AFGHVFUQ6USLQMUFSW2Q, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.847279324 (5h56m13.434335535s) [noise=true , address=45.91.116.12:4227 , piece_id=7RIB2VCODFSHTRTBGI3Z74IECO3ATQXSXW7CY3TCA43MZKSKEQYA, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.848559454 (5h56m13.435615675s) [noise=true , address=45.91.116.4:4940 , piece_id=UXKWVJQC6KYUHGXKHRYHJATOP5KQF4BY2LWNQJQEGFYKQEBT6GWA, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.852382881 (5h56m13.439439102s) [noise=true , address=45.91.116.9:4262 , piece_id=R7J2NVZJTXR7ATH2VUC75B2Z73TSJAQUBYJ3WE75RQ6AD3L3PSCQ, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.852577974 (5h56m13.439634195s) [noise=true , address=77.90.31.19:4842 , piece_id=ULCHXICRO3ULX7IEW6GVXHHGXT6F42N3WQDEEUVOSKCXID6F6XZQ, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.856642091 (5h56m13.443698302s) [noise=true , address=104.36.82.8:4971 , piece_id=2MZZBGEPYJSHOBAG4FUAU5NKKAOL72C2AWD3GZECQXGPMOG6TDMQ, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.856766534 (5h56m13.443822745s) [noise=true , address=77.90.31.10:4519 , piece_id=QF2SVPCWB5HG4S7GQL6NFHTYPVOHP4HHORRY2TMJSRXP5A3J43TQ, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.868085922 (5h56m13.455142143s) [noise=true , address=45.91.116.12:4225 , piece_id=MNJGBICBIG3HQXCO7B2C2ITRFSOW475RAGNVTAQNVFQYAJ7DOU5Q, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.868246762 (5h56m13.455302973s) [noise=true , address=104.36.82.2:4730 , piece_id=FKFZNBUC7L2WVTTFPNRSPJUWW6HAEE6CNUAYSHVDYXW5XENS462Q, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.877804453 (5h56m13.464860674s) [noise=true , address=104.36.82.11:4115 , piece_id=BALIRX6S5WW2CGSXSIFHWV2ABDH3ASPHR36YEYRUYTTGZCW6U6AQ, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.882357252 (5h56m13.469413462s) [noise=true , address=104.36.82.2:4731 , piece_id=L6W56WDLHKGHEPXBK7F2OYZ6CRM66ZVKG5RGJPH3FHC7RZZ3QGBA, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.887398722 (5h56m13.474454933s) [noise=true , address=104.36.82.14:4221 , piece_id=SBKEVRBOXXBFOVTZWPSDVDPXEDV6HZIVAV4MWFIYDMNEU643KBWA, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.887674137 (5h56m13.474730358s) [noise=true , address=104.36.82.10:4079 , piece_id=DHLJEMGMQUTVRWLFHTOF2YZFDYX3LU5FENJAS6JOW572GLJ2OHVQ, seg_pos={0 0}, upload=6077]: Done uploading piece. err: <nil> Aug 7 22:22:59.887689665 (5h56m13.474745877s) [seg_pos={0 0}, upload=6077]: Segment reached optimal threshold of 13 pieces. Aug 7 22:22:59.887775385 (5h56m13.474831606s) [noise=true , address=77.90.31.13:4628 , piece_id=BYQWTXNUD76XIWZZAUZNHDUX7F7CABMDCPK7M7QPDHS3CFBRBMVQ, seg_pos={0 0}, upload=6077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: expected piece hash; context canceled; EOF Aug 7 22:22:59.887795453 (5h56m13.474851674s) [noise=true , address=104.36.82.5:4840 , piece_id=6SYENOMYMIEYCEFSHJKMXOQE4RCDPDWBAHYDVRTXCQYWJGQDCCKA, seg_pos={0 0}, upload=6077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Aug 7 22:22:59.887879469 (5h56m13.47493569s) [noise=true , address=104.36.82.9:4006 , piece_id=MDXFW3M6TBQJXYHMBTPIOD5I5PK3NUHDRZ3EERNCRKFVMUPSW43Q, seg_pos={0 0}, upload=6077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc): protocol: write tcp 5.161.89.252:59712->104.36.82.9:4006: use of closed network connection; write tcp 5.161.89.252:59712->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:59712->104.36.82.9:4006: use of closed network connection Aug 7 22:22:59.887943669 (5h56m13.47499988s) [seg_pos={0 0}, upload=6077]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 22:22:59.888006888 (5h56m13.475063109s) [upload=6077]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:22:59.888010665 (5h56m13.475066886s) [upload=6077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:22:59.933305033 (5h56m13.520361243s) [upload=6077]: Done uploading segments... Aug 7 22:22:59.933317746 (5h56m13.520373958s) [upload=6077]: Upload finished. err: <nil> Aug 7 22:23:47.739190890 (5h57m1.326247131s) [upload=6091]: Starting upload Aug 7 22:23:47.739297178 (5h57m1.326353419s) [upload=6091]: Done starting upload Aug 7 22:23:47.739428053 (5h57m1.326484274s) [upload=6091]: Uploading segments... Aug 7 22:23:47.739777785 (5h57m1.326834025s) [upload=6091]: Got next segment. Inline: false Aug 7 22:23:47.739805196 (5h57m1.326861407s) [upload=6091]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:23:47.739811638 (5h57m1.326867849s) [upload=6091]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:24:17.722308743 (5h57m31.309364985s) [upload=6091]: Next segment err: context deadline exceeded Aug 7 22:24:17.722438996 (5h57m31.309495238s) [upload=6091]: Done uploading segments... Aug 7 22:24:17.722476005 (5h57m31.309532246s) [upload=6091]: Upload finished. err: context deadline exceeded Aug 7 22:24:18.676382471 (5h57m32.263438711s) [upload=6101]: Starting upload Aug 7 22:24:18.676482739 (5h57m32.263538979s) [upload=6101]: Done starting upload Aug 7 22:24:18.676619723 (5h57m32.263675964s) [upload=6101]: Uploading segments... Aug 7 22:24:18.676886642 (5h57m32.263942883s) [upload=6101]: Got next segment. Inline: false Aug 7 22:24:18.676922940 (5h57m32.263979181s) [upload=6101]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:24:18.676930835 (5h57m32.263987076s) [upload=6101]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:24:48.662510024 (5h58m2.249566255s) [upload=6101]: Next segment err: context deadline exceeded Aug 7 22:24:48.662708235 (5h58m2.249764465s) [upload=6101]: Done uploading segments... Aug 7 22:24:48.662735284 (5h58m2.249791526s) [upload=6101]: Upload finished. err: context deadline exceeded Aug 7 22:25:18.142081018 (5h58m31.729137259s) [upload=6117]: Starting upload Aug 7 22:25:18.142160586 (5h58m31.729216828s) [upload=6117]: Done starting upload Aug 7 22:25:18.142286381 (5h58m31.729342623s) [upload=6117]: Uploading segments... Aug 7 22:25:18.142594587 (5h58m31.729650828s) [upload=6117]: Got next segment. Inline: false Aug 7 22:25:18.142641995 (5h58m31.729698236s) [upload=6117]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:25:18.142647436 (5h58m31.729703676s) [upload=6117]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:25:48.125202589 (5h59m1.71225884s) [upload=6117]: Next segment err: context deadline exceeded Aug 7 22:25:48.125308446 (5h59m1.712364688s) [upload=6117]: Done uploading segments... Aug 7 22:25:48.125355273 (5h59m1.712411525s) [upload=6117]: Upload finished. err: context deadline exceeded Duration: 601.889838ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [...................................XXXXXXXXXXXXXXXXXXX..XXXX] duration: 30.001233927s FAIL (2025-08-07 22:30:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001252161s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 145.547885ms FAIL (2025-08-07 22:30:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ine Aug 7 21:59:56.515336538 (5h33m10.10239275s) [upload=5708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:59:56.574152557 (5h33m10.161208778s) [upload=5708]: Done uploading segments... Aug 7 21:59:56.574171943 (5h33m10.161228164s) [upload=5708]: Upload finished. err: <nil> Aug 7 22:00:36.611785100 (5h33m50.19884134s) [upload=5716]: Starting upload Aug 7 22:00:36.611856203 (5h33m50.198912444s) [upload=5716]: Done starting upload Aug 7 22:00:36.611959786 (5h33m50.199016027s) [upload=5716]: Uploading segments... Aug 7 22:00:36.612003668 (5h33m50.199059918s) [upload=5716]: Got next segment. Inline: true Aug 7 22:00:36.612009668 (5h33m50.19906591s) [upload=5716]: Waiting for error group managing segments... Aug 7 22:00:36.612057077 (5h33m50.199113319s) [upload=5716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:00:36.612082536 (5h33m50.199138776s) [upload=5716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:00:36.612086222 (5h33m50.199142473s) [upload=5716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:00:36.884781084 (5h33m50.471837294s) [upload=5716]: Done uploading segments... Aug 7 22:00:36.884795611 (5h33m50.471851832s) [upload=5716]: Upload finished. err: <nil> Aug 7 22:01:47.555449092 (5h35m1.142505313s) [upload=5736]: Starting upload Aug 7 22:01:47.555491781 (5h35m1.142548002s) [upload=5736]: Done starting upload Aug 7 22:01:47.555562123 (5h35m1.142618344s) [upload=5736]: Uploading segments... Aug 7 22:01:47.555595826 (5h35m1.142652047s) [upload=5736]: Got next segment. Inline: true Aug 7 22:01:47.555599363 (5h35m1.142655584s) [upload=5736]: Waiting for error group managing segments... Aug 7 22:01:47.555631602 (5h35m1.142687823s) [upload=5736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:47.555635390 (5h35m1.142691611s) [upload=5736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:47.555637894 (5h35m1.142694115s) [upload=5736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:47.686110188 (5h35m1.273166429s) [upload=5736]: Done uploading segments... Aug 7 22:01:47.686151165 (5h35m1.273207406s) [upload=5736]: Upload finished. err: <nil> Aug 7 22:02:22.678566055 (5h35m36.265622267s) [upload=5746]: Starting upload Aug 7 22:02:22.678633091 (5h35m36.265689303s) [upload=5746]: Done starting upload Aug 7 22:02:22.678684407 (5h35m36.265740617s) [upload=5746]: Uploading segments... Aug 7 22:02:22.678721757 (5h35m36.265777967s) [upload=5746]: Got next segment. Inline: true Aug 7 22:02:22.678725724 (5h35m36.265781935s) [upload=5746]: Waiting for error group managing segments... Aug 7 22:02:22.678759457 (5h35m36.265815668s) [upload=5746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:02:22.678764947 (5h35m36.265821168s) [upload=5746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:02:22.678768565 (5h35m36.265824826s) [upload=5746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:02:22.971349697 (5h35m36.558405918s) [upload=5746]: Done uploading segments... Aug 7 22:02:22.971373742 (5h35m36.558429963s) [upload=5746]: Upload finished. err: <nil> Aug 7 22:03:08.907032448 (5h36m22.494088689s) [upload=5757]: Starting upload Aug 7 22:03:08.907091528 (5h36m22.494147769s) [upload=5757]: Done starting upload Aug 7 22:03:08.907172368 (5h36m22.4942286s) [upload=5757]: Uploading segments... Aug 7 22:03:08.907209458 (5h36m22.494265699s) [upload=5757]: Got next segment. Inline: true Aug 7 22:03:08.907214758 (5h36m22.494271s) [upload=5757]: Waiting for error group managing segments... Aug 7 22:03:08.907255936 (5h36m22.494312176s) [upload=5757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:03:08.907261175 (5h36m22.494317416s) [upload=5757]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:03:08.907265904 (5h36m22.494322135s) [upload=5757]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:03:09.004207779 (5h36m22.59126404s) [upload=5757]: Done uploading segments... Aug 7 22:03:09.004232375 (5h36m22.591288627s) [upload=5757]: Upload finished. err: <nil> Aug 7 22:04:02.463086309 (5h37m16.050142521s) [upload=5775]: Starting upload Aug 7 22:04:02.463128347 (5h37m16.050184568s) [upload=5775]: Done starting upload Aug 7 22:04:02.463197818 (5h37m16.050254029s) [upload=5775]: Uploading segments... Aug 7 22:04:02.463223595 (5h37m16.050279816s) [upload=5775]: Got next segment. Inline: true Aug 7 22:04:02.463227652 (5h37m16.050283873s) [upload=5775]: Waiting for error group managing segments... Aug 7 22:04:02.463258120 (5h37m16.050314331s) [upload=5775]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:04:02.463261796 (5h37m16.050318008s) [upload=5775]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:04:02.463264733 (5h37m16.050320954s) [upload=5775]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:04:02.530547234 (5h37m16.117603445s) [upload=5775]: Done uploading segments... Aug 7 22:04:02.530562512 (5h37m16.117618733s) [upload=5775]: Upload finished. err: <nil> Aug 7 22:05:10.744065032 (5h38m24.331121272s) [upload=5796]: Starting upload Aug 7 22:05:10.744195716 (5h38m24.331251957s) [upload=5796]: Done starting upload Aug 7 22:05:10.744327762 (5h38m24.331384002s) [upload=5796]: Uploading segments... Aug 7 22:05:10.744369549 (5h38m24.331425791s) [upload=5796]: Got next segment. Inline: true Aug 7 22:05:10.744388655 (5h38m24.331444897s) [upload=5796]: Waiting for error group managing segments... Aug 7 22:05:10.744449028 (5h38m24.331505259s) [upload=5796]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:05:10.744454398 (5h38m24.331510629s) [upload=5796]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:05:10.744457633 (5h38m24.331513875s) [upload=5796]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:05:10.852991578 (5h38m24.440047789s) [upload=5796]: Done uploading segments... Aug 7 22:05:10.853004571 (5h38m24.440060782s) [upload=5796]: Upload finished. err: <nil> Aug 7 22:05:45.060720865 (5h38m58.647777086s) [upload=5806]: Starting upload Aug 7 22:05:45.060780046 (5h38m58.647836267s) [upload=5806]: Done starting upload Aug 7 22:05:45.060842121 (5h38m58.647898333s) [upload=5806]: Uploading segments... Aug 7 22:05:45.060876836 (5h38m58.647933057s) [upload=5806]: Got next segment. Inline: true Aug 7 22:05:45.060882878 (5h38m58.647939099s) [upload=5806]: Waiting for error group managing segments... Aug 7 22:05:45.060927901 (5h38m58.647984112s) [upload=5806]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:05:45.060933843 (5h38m58.647990064s) [upload=5806]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:05:45.060936187 (5h38m58.647992408s) [upload=5806]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:05:45.328399453 (5h38m58.915455674s) [upload=5806]: Done uploading segments... Aug 7 22:05:45.328433367 (5h38m58.915489588s) [upload=5806]: Upload finished. err: <nil> Aug 7 22:06:12.931860875 (5h39m26.518917096s) [upload=5815]: Starting upload Aug 7 22:06:12.931905979 (5h39m26.51896219s) [upload=5815]: Done starting upload Aug 7 22:06:12.931964518 (5h39m26.51902073s) [upload=5815]: Uploading segments... Aug 7 22:06:12.931990617 (5h39m26.519046827s) [upload=5815]: Got next segment. Inline: true Aug 7 22:06:12.931994224 (5h39m26.519050434s) [upload=5815]: Waiting for error group managing segments... Aug 7 22:06:12.932024821 (5h39m26.519081042s) [upload=5815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:06:12.932028408 (5h39m26.519084618s) [upload=5815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:06:12.932030893 (5h39m26.519087103s) [upload=5815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:06:13.014995076 (5h39m26.602051317s) [upload=5815]: Done uploading segments... Aug 7 22:06:13.015014863 (5h39m26.602071114s) [upload=5815]: Upload finished. err: <nil> Aug 7 22:07:16.916734820 (5h40m30.503791062s) [upload=5832]: Starting upload Aug 7 22:07:16.916800323 (5h40m30.503856563s) [upload=5832]: Done starting upload Aug 7 22:07:16.916864883 (5h40m30.503921123s) [upload=5832]: Uploading segments... Aug 7 22:07:16.916903304 (5h40m30.503959546s) [upload=5832]: Got next segment. Inline: true Aug 7 22:07:16.916908815 (5h40m30.503965056s) [upload=5832]: Waiting for error group managing segments... Aug 7 22:07:16.916952356 (5h40m30.504008598s) [upload=5832]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:07:16.916957657 (5h40m30.504013897s) [upload=5832]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:07:16.916961754 (5h40m30.504017995s) [upload=5832]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:07:16.988776316 (5h40m30.575832566s) [upload=5832]: Done uploading segments... Aug 7 22:07:16.988801573 (5h40m30.575857824s) [upload=5832]: Upload finished. err: <nil> Aug 7 22:08:29.650294124 (5h41m43.237350365s) [upload=5854]: Starting upload Aug 7 22:08:29.650357383 (5h41m43.237413623s) [upload=5854]: Done starting upload Aug 7 22:08:29.650418325 (5h41m43.237474567s) [upload=5854]: Uploading segments... Aug 7 22:08:29.650468490 (5h41m43.237524731s) [upload=5854]: Got next segment. Inline: true Aug 7 22:08:29.650474350 (5h41m43.237530592s) [upload=5854]: Waiting for error group managing segments... Aug 7 22:08:29.650515618 (5h41m43.237571859s) [upload=5854]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:08:29.650522029 (5h41m43.237578271s) [upload=5854]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:08:29.650525556 (5h41m43.237581797s) [upload=5854]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:08:29.706715553 (5h41m43.293771793s) [upload=5854]: Done uploading segments... Aug 7 22:08:29.706731001 (5h41m43.293787243s) [upload=5854]: Upload finished. err: metaclient: Too Many Requests Duration:* 145.571979ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 22 out of 60 is failed (threshold=3) [XXXXXX.XX......X.............................XXXXXXXXXXXXX..] duration: 927.276998ms FAIL (2025-08-07 22:30:16 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== [upload=6089]: Done starting upload Aug 7 22:23:43.961837605 (5h56m57.548893826s) [upload=6089]: Uploading segments... Aug 7 22:23:43.962138106 (5h56m57.549194347s) [upload=6089]: Got next segment. Inline: false Aug 7 22:23:43.962204581 (5h56m57.549260822s) [upload=6089]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:23:43.962209980 (5h56m57.549266222s) [upload=6089]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:24:12.375269699 (5h57m25.96232592s) [seg_pos={0 0}, upload=6089]: Begin upload segment... Aug 7 22:24:12.375313250 (5h57m25.962369471s) [seg_pos={0 0}, upload=6089]: Done begin upload segment. Aug 7 22:24:12.375339780 (5h57m25.962396031s) [noise=true , address=193.149.164.143:4698 , piece_id=AV2DCKYWWT5N4PL57WTXW45JNFYOCRLLEY6YSBSWOAQAPHAVXJQQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.375356050 (5h57m25.962412261s) [noise=true , address=104.36.82.53:4156 , piece_id=NBWZ4FB4S7CTDN2OTWTDKSH2R7DJN66P7GRTQTQ5FJZFQCZLDAQQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.375715481 (5h57m25.962771693s) [noise=true , address=104.36.82.58:4280 , piece_id=W2A65H4FIUOCUSMSVCANR2OPVBUFO4YNAFZV3KCASSCSPCHONGBA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.375724769 (5h57m25.96278099s) [noise=true , address=104.36.82.59:4333 , piece_id=JONL6XDWEUYMHVDPJVB54PE5HCYWN7KRO3FSNAMPJWCEP6QBKYVQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376022925 (5h57m25.963079146s) [noise=true , address=45.91.116.9:4270 , piece_id=CMJ7K7XDV44UUOWXU6NOUNMWZGLOEZKPTLVFMTNWIR6OK52TOEKQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376075263 (5h57m25.963131475s) [noise=true , address=104.36.82.36:4652 , piece_id=ZDJ2LSAX25MBAAW4PW745SX7J7X3S4PAGXEUS36T43KTM3LZ2WRQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376490167 (5h57m25.963546388s) [noise=true , address=104.36.82.45:4921 , piece_id=DCEBY7YTNAO4UR6YZ5LC5W27INFXZQ26BPJ7BHPMXY26P5BDL23Q, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376527327 (5h57m25.963583599s) [noise=true , address=104.36.82.12:4169 , piece_id=XPNXIWBPTKCEEQTVQ3BAQ3HCLQE5U5BTRIAR3FQ6O4OQJZMCJ6ZA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376755913 (5h57m25.963812125s) [noise=true , address=104.36.82.62:4411 , piece_id=3TMITFZMXF5ESOCJNLFU6EIT7CFUYULYZ6FJUQ6X7T2WBEQMKWFQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376864006 (5h57m25.963920216s) [noise=true , address=104.36.82.14:4243 , piece_id=WKT5D56ZXVSJJHVR3GB6N5XR6CGU4VW2XK2YTMPBZOH4S2ZPOVCA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.376979641 (5h57m25.964035862s) [noise=true , address=104.36.82.9:4031 , piece_id=UOVVYSA77DQOK53ATDDXV56PLEVS6GBQ5HMZEKCEURYBPRCXNYPA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.377222374 (5h57m25.964278595s) [noise=true , address=104.36.82.52:4100 , piece_id=7RUPILCVG3GL3KB4CS25YCYKDB4C6HFT6TGZWZIDC5EHBI2SD55Q, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.377246499 (5h57m25.96430271s) [noise=true , address=79.127.201.215:4343 , piece_id=N3KGRRGXWJLCJGYSLPOLOIYZHP7AFWV6VBCHVBETDYOQBIXIWPMQ, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.377620267 (5h57m25.964676488s) [noise=true , address=104.36.82.35:4597 , piece_id=6UMSPJQOVLEDWMIQMLAJLZ2AY6Y5XLMODCN3V63EB5JNJS3EC7BA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.377642138 (5h57m25.964698349s) [noise=true , address=104.36.82.16:4300 , piece_id=RJRU7SETYXWMQK7GEE3HBD2XMEVSI3UJ6ZDCIWVW4266QSNAOPZA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.377932059 (5h57m25.96498828s) [noise=true , address=104.36.82.47:4544 , piece_id=DVQRASYHRKPFEONZEWJ3MBVM7O5764NPWYADLMG4ARAH5PBD55MA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.377950764 (5h57m25.965006985s) [noise=true , address=104.36.82.21:4478 , piece_id=5UO756WUOKCS52UJI2NIQFUSU7W6NBW34MGFH6YZX27HKWY5243A, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.378280479 (5h57m25.9653367s) [noise=true , address=104.36.82.37:4656 , piece_id=A25QOY4UFT7RRRY6PVAS3NNML7PGJM4OHDH56PF2HZHKJ7VOPURA, seg_pos={0 0}, upload=6089]: Uploading piece... Aug 7 22:24:12.468526485 (5h57m26.055582696s) [upload=6089]: Next returned nil segment Aug 7 22:24:12.468535081 (5h57m26.055591292s) [upload=6089]: Waiting for error group managing segments... Aug 7 22:24:12.483383738 (5h57m26.070439949s) [noise=true , address=104.36.82.37:4656 , piece_id=A25QOY4UFT7RRRY6PVAS3NNML7PGJM4OHDH56PF2HZHKJ7VOPURA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.500821799 (5h57m26.08787801s) [noise=true , address=104.36.82.16:4300 , piece_id=RJRU7SETYXWMQK7GEE3HBD2XMEVSI3UJ6ZDCIWVW4266QSNAOPZA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.504410257 (5h57m26.091466458s) [noise=true , address=45.91.116.9:4270 , piece_id=CMJ7K7XDV44UUOWXU6NOUNMWZGLOEZKPTLVFMTNWIR6OK52TOEKQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.511708921 (5h57m26.098765132s) [noise=true , address=104.36.82.35:4597 , piece_id=6UMSPJQOVLEDWMIQMLAJLZ2AY6Y5XLMODCN3V63EB5JNJS3EC7BA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.523758022 (5h57m26.110814233s) [noise=true , address=104.36.82.12:4169 , piece_id=XPNXIWBPTKCEEQTVQ3BAQ3HCLQE5U5BTRIAR3FQ6O4OQJZMCJ6ZA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.533614250 (5h57m26.120670461s) [noise=true , address=104.36.82.52:4100 , piece_id=7RUPILCVG3GL3KB4CS25YCYKDB4C6HFT6TGZWZIDC5EHBI2SD55Q, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.557032764 (5h57m26.144088985s) [noise=true , address=104.36.82.21:4478 , piece_id=5UO756WUOKCS52UJI2NIQFUSU7W6NBW34MGFH6YZX27HKWY5243A, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.582698011 (5h57m26.169754223s) [noise=true , address=104.36.82.53:4156 , piece_id=NBWZ4FB4S7CTDN2OTWTDKSH2R7DJN66P7GRTQTQ5FJZFQCZLDAQQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.609343038 (5h57m26.196399279s) [noise=true , address=79.127.201.215:4343 , piece_id=N3KGRRGXWJLCJGYSLPOLOIYZHP7AFWV6VBCHVBETDYOQBIXIWPMQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.639134328 (5h57m26.226190538s) [noise=true , address=104.36.82.36:4652 , piece_id=ZDJ2LSAX25MBAAW4PW745SX7J7X3S4PAGXEUS36T43KTM3LZ2WRQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:12.956879031 (5h57m26.543935272s) [noise=true , address=193.149.164.143:4698 , piece_id=AV2DCKYWWT5N4PL57WTXW45JNFYOCRLLEY6YSBSWOAQAPHAVXJQQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:13.036416979 (5h57m26.623473231s) [noise=true , address=104.36.82.9:4031 , piece_id=UOVVYSA77DQOK53ATDDXV56PLEVS6GBQ5HMZEKCEURYBPRCXNYPA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:13.172901316 (5h57m26.759957557s) [noise=true , address=104.36.82.47:4544 , piece_id=DVQRASYHRKPFEONZEWJ3MBVM7O5764NPWYADLMG4ARAH5PBD55MA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:13.202754351 (5h57m26.789810592s) [noise=true , address=104.36.82.14:4243 , piece_id=WKT5D56ZXVSJJHVR3GB6N5XR6CGU4VW2XK2YTMPBZOH4S2ZPOVCA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: <nil> Aug 7 22:24:13.202797742 (5h57m26.789853984s) [seg_pos={0 0}, upload=6089]: Segment reached optimal threshold of 14 pieces. Aug 7 22:24:13.203007333 (5h57m26.790063585s) [noise=true , address=104.36.82.58:4280 , piece_id=W2A65H4FIUOCUSMSVCANR2OPVBUFO4YNAFZV3KCASSCSPCHONGBA, seg_pos={0 0}, upload=6089]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fgHaEFKK4CeGg4x3vywTuYfT3kmac31xqg2QpsahQNXmvjhW8): protocol: expected piece hash; context canceled; EOF Aug 7 22:24:13.203190164 (5h57m26.790246415s) [noise=true , address=104.36.82.59:4333 , piece_id=JONL6XDWEUYMHVDPJVB54PE5HCYWN7KRO3FSNAMPJWCEP6QBKYVQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K7U6NXcB6SFBtQFrG7DXt2skACr3u5DX2QsrnToi8EKywS9TN): protocol: write tcp 5.161.89.252:63730->104.36.82.59:4333: use of closed network connection; write tcp 5.161.89.252:63730->104.36.82.59:4333: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:63730->104.36.82.59:4333: use of closed network connection Aug 7 22:24:13.203221153 (5h57m26.790277394s) [noise=true , address=104.36.82.45:4921 , piece_id=DCEBY7YTNAO4UR6YZ5LC5W27INFXZQ26BPJ7BHPMXY26P5BDL23Q, seg_pos={0 0}, upload=6089]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bBRKXiuSN2cw6KrdP8q4Dsf4MiFCUSgMEGjqG2dKL9RLTQra1): protocol: expected piece hash; context canceled; EOF Aug 7 22:24:13.203424783 (5h57m26.790481023s) [noise=true , address=104.36.82.62:4411 , piece_id=3TMITFZMXF5ESOCJNLFU6EIT7CFUYULYZ6FJUQ6X7T2WBEQMKWFQ, seg_pos={0 0}, upload=6089]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DT68m6J4PpDk3XUDhDaaqV46NLKVz4A23716JLFxEAh122GeR): protocol: context canceled; context canceled Aug 7 22:24:13.203452675 (5h57m26.790508916s) [seg_pos={0 0}, upload=6089]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 22:24:13.203591695 (5h57m26.790647935s) [upload=6089]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:24:13.203602313 (5h57m26.790658565s) [upload=6089]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:29:03.852878645 (6h2m17.439934866s) [upload=6089]: Done uploading segments... Aug 7 22:29:03.852904053 (6h2m17.439960294s) [upload=6089]: Upload finished. err: metaclient: context deadline exceeded Duration: 927.301664ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 110.670972ms remote: "tcp:136.0.77.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 22:30:13 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 110.695709ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 22 out of 60 is failed (threshold=3) [XXXXX.X.XX...............................XXXXXXXXXXX.X.XX...] duration: 188.543993ms FAIL (2025-08-07 22:30:06 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 04]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:19:04.377299713 (5h52m17.964355924s) [upload=6018]: Starting upload Aug 7 22:19:04.377350166 (5h52m17.964406387s) [upload=6018]: Done starting upload Aug 7 22:19:04.377401523 (5h52m17.964457744s) [upload=6018]: Uploading segments... Aug 7 22:19:04.377446897 (5h52m17.964503118s) [upload=6018]: Got next segment. Inline: true Aug 7 22:19:04.377453410 (5h52m17.96450962s) [upload=6018]: Waiting for error group managing segments... Aug 7 22:19:04.377487142 (5h52m17.964543354s) [upload=6018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:19:04.377491361 (5h52m17.964547582s) [upload=6018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:19:04.377493875 (5h52m17.964550086s) [upload=6018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:20:04.361964803 (5h53m17.949021024s) [upload=6018]: Done uploading segments... Aug 7 22:20:04.361979240 (5h53m17.949035461s) [upload=6018]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:20:11.847632631 (5h53m25.434688843s) [upload=6035]: Starting upload Aug 7 22:20:11.847694537 (5h53m25.434750747s) [upload=6035]: Done starting upload Aug 7 22:20:11.847740192 (5h53m25.434796403s) [upload=6035]: Uploading segments... Aug 7 22:20:11.847772231 (5h53m25.434828442s) [upload=6035]: Got next segment. Inline: true Aug 7 22:20:11.847776860 (5h53m25.434833071s) [upload=6035]: Waiting for error group managing segments... Aug 7 22:20:11.847824679 (5h53m25.434880891s) [upload=6035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:20:11.847830130 (5h53m25.434886341s) [upload=6035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:20:11.847833205 (5h53m25.434889417s) [upload=6035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:21:11.836000640 (5h54m25.42305688s) [upload=6035]: Done uploading segments... Aug 7 22:21:11.836026087 (5h54m25.423082328s) [upload=6035]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:21:11.965120248 (5h54m25.552176489s) [upload=6050]: Starting upload Aug 7 22:21:11.965231515 (5h54m25.552287757s) [upload=6050]: Done starting upload Aug 7 22:21:11.965301156 (5h54m25.552357397s) [upload=6050]: Uploading segments... Aug 7 22:21:11.965348053 (5h54m25.552404304s) [upload=6050]: Got next segment. Inline: true Aug 7 22:21:11.965353994 (5h54m25.552410235s) [upload=6050]: Waiting for error group managing segments... Aug 7 22:21:11.965399049 (5h54m25.55245529s) [upload=6050]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:21:11.965405170 (5h54m25.552461411s) [upload=6050]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:21:11.965408957 (5h54m25.552465198s) [upload=6050]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:22:11.952333050 (5h55m25.539389261s) [upload=6050]: Done uploading segments... Aug 7 22:22:11.952348108 (5h55m25.539404329s) [upload=6050]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:22:12.125652873 (5h55m25.712709094s) [upload=6063]: Starting upload Aug 7 22:22:12.125702476 (5h55m25.712758688s) [upload=6063]: Done starting upload Aug 7 22:22:12.125747020 (5h55m25.712803241s) [upload=6063]: Uploading segments... Aug 7 22:22:12.125776895 (5h55m25.712833116s) [upload=6063]: Got next segment. Inline: true Aug 7 22:22:12.125781404 (5h55m25.712837625s) [upload=6063]: Waiting for error group managing segments... Aug 7 22:22:12.125814315 (5h55m25.712870536s) [upload=6063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:22:12.125818052 (5h55m25.712874263s) [upload=6063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:22:12.125820397 (5h55m25.712876618s) [upload=6063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:23:12.024254277 (5h56m25.611310498s) [upload=6063]: Done uploading segments... Aug 7 22:23:12.024289072 (5h56m25.611345283s) [upload=6063]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:23:12.175471355 (5h56m25.762527616s) [upload=6082]: Starting upload Aug 7 22:23:12.175573175 (5h56m25.762629466s) [upload=6082]: Done starting upload Aug 7 22:23:12.175670356 (5h56m25.762726627s) [upload=6082]: Uploading segments... Aug 7 22:23:12.175760093 (5h56m25.762816364s) [upload=6082]: Got next segment. Inline: true Aug 7 22:23:12.175767677 (5h56m25.762823919s) [upload=6082]: Waiting for error group managing segments... Aug 7 22:23:12.175819345 (5h56m25.762875585s) [upload=6082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:23:12.175825105 (5h56m25.762881346s) [upload=6082]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:23:12.175828732 (5h56m25.762884973s) [upload=6082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:24:12.154559214 (5h57m25.741615455s) [upload=6082]: Done uploading segments... Aug 7 22:24:12.154578851 (5h57m25.741635092s) [upload=6082]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:24:12.253523032 (5h57m25.840579253s) [upload=6098]: Starting upload Aug 7 22:24:12.253572454 (5h57m25.840628665s) [upload=6098]: Done starting upload Aug 7 22:24:12.253615034 (5h57m25.840671245s) [upload=6098]: Uploading segments... Aug 7 22:24:12.253654808 (5h57m25.840711029s) [upload=6098]: Got next segment. Inline: true Aug 7 22:24:12.253658636 (5h57m25.840714847s) [upload=6098]: Waiting for error group managing segments... Aug 7 22:24:12.253689122 (5h57m25.840745343s) [upload=6098]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:24:12.253692969 (5h57m25.84074918s) [upload=6098]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:12.253695303 (5h57m25.840751524s) [upload=6098]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:24:12.467343237 (5h57m26.054399448s) [upload=6098]: Done uploading segments... Aug 7 22:24:12.467365428 (5h57m26.05442164s) [upload=6098]: Upload finished. err: <nil> Aug 7 22:24:33.085790933 (5h57m46.672847184s) [upload=6106]: Starting upload Aug 7 22:24:33.085870281 (5h57m46.672926522s) [upload=6106]: Done starting upload Aug 7 22:24:33.085932668 (5h57m46.67298892s) [upload=6106]: Uploading segments... Aug 7 22:24:33.085994053 (5h57m46.673050293s) [upload=6106]: Got next segment. Inline: true Aug 7 22:24:33.086000064 (5h57m46.673056304s) [upload=6106]: Waiting for error group managing segments... Aug 7 22:24:33.086045709 (5h57m46.67310195s) [upload=6106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:24:33.086051168 (5h57m46.67310741s) [upload=6106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:33.086054985 (5h57m46.673111227s) [upload=6106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:33.072711385 (5h58m46.659767636s) [upload=6106]: Done uploading segments... Aug 7 22:25:33.072753384 (5h58m46.659809634s) [upload=6106]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:25:33.160012555 (5h58m46.747068795s) [upload=6120]: Starting upload Aug 7 22:25:33.160101981 (5h58m46.747158222s) [upload=6120]: Done starting upload Aug 7 22:25:33.160199473 (5h58m46.747255715s) [upload=6120]: Uploading segments... Aug 7 22:25:33.160243596 (5h58m46.747299837s) [upload=6120]: Got next segment. Inline: true Aug 7 22:25:33.160249046 (5h58m46.747305287s) [upload=6120]: Waiting for error group managing segments... Aug 7 22:25:33.160294231 (5h58m46.747350482s) [upload=6120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:25:33.160300012 (5h58m46.747356252s) [upload=6120]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:25:33.160303619 (5h58m46.747359859s) [upload=6120]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:33.228188865 (5h58m46.815245086s) [upload=6120]: Done uploading segments... Aug 7 22:25:33.228202571 (5h58m46.815258782s) [upload=6120]: Upload finished. err: <nil> Aug 7 22:26:14.972993755 (5h59m28.560049967s) [upload=6132]: Starting upload Aug 7 22:26:14.973047095 (5h59m28.560103307s) [upload=6132]: Done starting upload Aug 7 22:26:14.973101456 (5h59m28.560157677s) [upload=6132]: Uploading segments... Aug 7 22:26:14.973137704 (5h59m28.560193925s) [upload=6132]: Got next segment. Inline: true Aug 7 22:26:14.973141382 (5h59m28.560197592s) [upload=6132]: Waiting for error group managing segments... Aug 7 22:26:14.973187948 (5h59m28.560244158s) [upload=6132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:26:14.973191995 (5h59m28.560248217s) [upload=6132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:26:14.973195201 (5h59m28.560251423s) [upload=6132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:14.950324988 (6h0m28.537381229s) [upload=6132]: Done uploading segments... Aug 7 22:27:14.950345787 (6h0m28.537402028s) [upload=6132]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:27:22.925128081 (6h0m36.512184292s) [upload=6146]: Starting upload Aug 7 22:27:22.925194244 (6h0m36.512250465s) [upload=6146]: Done starting upload Aug 7 22:27:22.925235571 (6h0m36.512291783s) [upload=6146]: Uploading segments... Aug 7 22:27:22.925260508 (6h0m36.512316729s) [upload=6146]: Got next segment. Inline: true Aug 7 22:27:22.925264486 (6h0m36.512320707s) [upload=6146]: Waiting for error group managing segments... Aug 7 22:27:22.925305752 (6h0m36.512361973s) [upload=6146]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:22.925309750 (6h0m36.512365971s) [upload=6146]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:22.925312044 (6h0m36.512368265s) [upload=6146]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:28:22.913115749 (6h1m36.50017199s) [upload=6146]: Done uploading segments... Aug 7 22:28:22.913136929 (6h1m36.500193169s) [upload=6146]: Upload finished. err: metaclient: context deadline exceeded Duration: 188.571114ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 23 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXX..................................XXXXXXXX.X..] duration: 1.779202111s FAIL (2025-08-07 22:30:03 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.779230242s core uplink us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 6 out of 60 is failed (threshold=3) [...................................................XX.XXXX..] duration: 38.864275ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 22:30:00 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 38.886948ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 27 out of 60 is failed (threshold=3) [.XXXXX.............................XXXXXXXXXXXXXXXX.XXXXX.X.] duration: 165.246976ms FAIL (2025-08-07 22:29:53 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 420501s) [upload=6055]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:22:06.462569787 (5h55m20.049626008s) [upload=6061]: Starting upload Aug 7 22:22:06.462655577 (5h55m20.049711788s) [upload=6061]: Done starting upload Aug 7 22:22:06.462750936 (5h55m20.049807157s) [upload=6061]: Uploading segments... Aug 7 22:22:06.462782464 (5h55m20.049838675s) [upload=6061]: Got next segment. Inline: true Aug 7 22:22:06.462786181 (5h55m20.049842402s) [upload=6061]: Waiting for error group managing segments... Aug 7 22:22:06.462817791 (5h55m20.049874001s) [upload=6061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:22:06.462821648 (5h55m20.049877869s) [upload=6061]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:22:06.462823982 (5h55m20.049880203s) [upload=6061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:22:36.449657257 (5h55m50.036713498s) [upload=6061]: Done uploading segments... Aug 7 22:22:36.449678196 (5h55m50.036734448s) [upload=6061]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:22:59.911739779 (5h56m13.498795989s) [upload=6078]: Starting upload Aug 7 22:22:59.911786736 (5h56m13.498842957s) [upload=6078]: Done starting upload Aug 7 22:22:59.911827472 (5h56m13.498883683s) [upload=6078]: Uploading segments... Aug 7 22:22:59.911861074 (5h56m13.498917295s) [upload=6078]: Got next segment. Inline: true Aug 7 22:22:59.911864401 (5h56m13.498920622s) [upload=6078]: Waiting for error group managing segments... Aug 7 22:22:59.911889107 (5h56m13.498945319s) [upload=6078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:22:59.911893005 (5h56m13.498949226s) [upload=6078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:22:59.911894898 (5h56m13.498951119s) [upload=6078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:23:29.900458747 (5h56m43.487514958s) [upload=6078]: Done uploading segments... Aug 7 22:23:29.900473835 (5h56m43.487530047s) [upload=6078]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:23:58.370463254 (5h57m11.957519465s) [upload=6094]: Starting upload Aug 7 22:23:58.370512907 (5h57m11.957569128s) [upload=6094]: Done starting upload Aug 7 22:23:58.370553031 (5h57m11.957609243s) [upload=6094]: Uploading segments... Aug 7 22:23:58.370578639 (5h57m11.95763486s) [upload=6094]: Got next segment. Inline: true Aug 7 22:23:58.370582427 (5h57m11.957638648s) [upload=6094]: Waiting for error group managing segments... Aug 7 22:23:58.370610951 (5h57m11.957667172s) [upload=6094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:23:58.370615178 (5h57m11.957671399s) [upload=6094]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:23:58.370617392 (5h57m11.957673613s) [upload=6094]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:23:58.450972793 (5h57m12.038029004s) [upload=6094]: Done uploading segments... Aug 7 22:23:58.450986078 (5h57m12.038042289s) [upload=6094]: Upload finished. err: <nil> Aug 7 22:24:40.794473113 (5h57m54.381529325s) [upload=6107]: Starting upload Aug 7 22:24:40.794523428 (5h57m54.381579639s) [upload=6107]: Done starting upload Aug 7 22:24:40.794565206 (5h57m54.381621417s) [upload=6107]: Uploading segments... Aug 7 22:24:40.794591815 (5h57m54.381648026s) [upload=6107]: Got next segment. Inline: true Aug 7 22:24:40.794595622 (5h57m54.381651833s) [upload=6107]: Waiting for error group managing segments... Aug 7 22:24:40.794624367 (5h57m54.381680577s) [upload=6107]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:24:40.794628323 (5h57m54.381684535s) [upload=6107]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:40.794630638 (5h57m54.381686849s) [upload=6107]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:10.774549311 (5h58m24.361605533s) [upload=6107]: Done uploading segments... Aug 7 22:25:10.774570019 (5h58m24.361626231s) [upload=6107]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:25:10.855666099 (5h58m24.442722339s) [upload=6114]: Starting upload Aug 7 22:25:10.855752680 (5h58m24.442808921s) [upload=6114]: Done starting upload Aug 7 22:25:10.855822740 (5h58m24.442878991s) [upload=6114]: Uploading segments... Aug 7 22:25:10.855866683 (5h58m24.442922923s) [upload=6114]: Got next segment. Inline: true Aug 7 22:25:10.855872564 (5h58m24.442928805s) [upload=6114]: Waiting for error group managing segments... Aug 7 22:25:10.855966008 (5h58m24.443022249s) [upload=6114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:25:10.855972971 (5h58m24.443029212s) [upload=6114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:25:10.855976568 (5h58m24.443032819s) [upload=6114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:40.842457902 (5h58m54.429514153s) [upload=6114]: Done uploading segments... Aug 7 22:25:40.842479021 (5h58m54.429535272s) [upload=6114]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:25:40.940807525 (5h58m54.527863766s) [upload=6122]: Starting upload Aug 7 22:25:40.940896992 (5h58m54.527953233s) [upload=6122]: Done starting upload Aug 7 22:25:40.940964388 (5h58m54.528020628s) [upload=6122]: Uploading segments... Aug 7 22:25:40.941010434 (5h58m54.528066675s) [upload=6122]: Got next segment. Inline: true Aug 7 22:25:40.941016375 (5h58m54.528072616s) [upload=6122]: Waiting for error group managing segments... Aug 7 22:25:40.941060899 (5h58m54.52811715s) [upload=6122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:25:40.941068091 (5h58m54.528124332s) [upload=6122]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:25:40.941073060 (5h58m54.528129311s) [upload=6122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:26:10.929528068 (5h59m24.516584319s) [upload=6122]: Done uploading segments... Aug 7 22:26:10.929548557 (5h59m24.516604797s) [upload=6122]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:26:11.008231659 (5h59m24.59528788s) [upload=6129]: Starting upload Aug 7 22:26:11.008286501 (5h59m24.595342713s) [upload=6129]: Done starting upload Aug 7 22:26:11.008333198 (5h59m24.595389419s) [upload=6129]: Uploading segments... Aug 7 22:26:11.008364838 (5h59m24.595421059s) [upload=6129]: Got next segment. Inline: true Aug 7 22:26:11.008368595 (5h59m24.595424816s) [upload=6129]: Waiting for error group managing segments... Aug 7 22:26:11.008399943 (5h59m24.595456164s) [upload=6129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:26:11.008403599 (5h59m24.59545981s) [upload=6129]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:26:11.008406565 (5h59m24.595462786s) [upload=6129]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:26:40.996305184 (5h59m54.583361385s) [upload=6129]: Done uploading segments... Aug 7 22:26:40.996323769 (5h59m54.58337998s) [upload=6129]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:27:14.829073707 (6h0m28.416129948s) [upload=6144]: Starting upload Aug 7 22:27:14.829153145 (6h0m28.416209386s) [upload=6144]: Done starting upload Aug 7 22:27:14.829247050 (6h0m28.416303291s) [upload=6144]: Uploading segments... Aug 7 22:27:14.829299899 (6h0m28.41635614s) [upload=6144]: Got next segment. Inline: true Aug 7 22:27:14.829306702 (6h0m28.416362943s) [upload=6144]: Waiting for error group managing segments... Aug 7 22:27:14.829350864 (6h0m28.416407105s) [upload=6144]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:14.829357416 (6h0m28.416413657s) [upload=6144]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:14.829361143 (6h0m28.416417384s) [upload=6144]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:44.808549878 (6h0m58.395606099s) [upload=6144]: Done uploading segments... Aug 7 22:27:44.808564877 (6h0m58.395621098s) [upload=6144]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:28:23.619901331 (6h1m37.206957573s) [upload=6162]: Starting upload Aug 7 22:28:23.620017008 (6h1m37.207073248s) [upload=6162]: Done starting upload Aug 7 22:28:23.620072551 (6h1m37.207128791s) [upload=6162]: Uploading segments... Aug 7 22:28:23.620125059 (6h1m37.2071813s) [upload=6162]: Got next segment. Inline: true Aug 7 22:28:23.620132432 (6h1m37.207188672s) [upload=6162]: Waiting for error group managing segments... Aug 7 22:28:23.620198335 (6h1m37.207254577s) [upload=6162]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:28:23.620204407 (6h1m37.207260647s) [upload=6162]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:28:23.620208153 (6h1m37.207264395s) [upload=6162]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:28:23.683041871 (6h1m37.270098112s) [upload=6162]: Done uploading segments... Aug 7 22:28:23.683055537 (6h1m37.270111778s) [upload=6162]: Upload finished. err: <nil> Aug 7 22:28:40.804037302 (6h1m54.391093523s) [upload=6165]: Starting upload Aug 7 22:28:40.804089460 (6h1m54.391145671s) [upload=6165]: Done starting upload Aug 7 22:28:40.804134294 (6h1m54.391190515s) [upload=6165]: Uploading segments... Aug 7 22:28:40.804181422 (6h1m54.391237632s) [upload=6165]: Got next segment. Inline: true Aug 7 22:28:40.804185379 (6h1m54.3912416s) [upload=6165]: Waiting for error group managing segments... Aug 7 22:28:40.804217198 (6h1m54.391273419s) [upload=6165]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:28:40.804220915 (6h1m54.391277127s) [upload=6165]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:28:40.804223370 (6h1m54.391279591s) [upload=6165]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:29:10.793781791 (6h2m24.380838023s) [upload=6165]: Done uploading segments... Aug 7 22:29:10.793803392 (6h2m24.380859603s) [upload=6165]: Upload finished. err: metaclient: context deadline exceeded Duration: 165.27107ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 21 out of 60 is failed (threshold=3) [X.....X..............................XXXXXXXXXXXXXXXXX..XX..] duration: 190.821254ms FAIL (2025-08-07 22:29:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 190.839758ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 22 out of 60 is failed (threshold=3) [X..................................XXXXXXXXXXXXXXXX.X.XXXX..] duration: 287.312027ms FAIL (2025-08-07 22:29:49 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 287.328938ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 1.501393473s FAIL (2025-08-07 22:29:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ine Aug 7 21:59:56.515336538 (5h33m10.10239275s) [upload=5708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:59:56.574152557 (5h33m10.161208778s) [upload=5708]: Done uploading segments... Aug 7 21:59:56.574171943 (5h33m10.161228164s) [upload=5708]: Upload finished. err: <nil> Aug 7 22:00:36.611785100 (5h33m50.19884134s) [upload=5716]: Starting upload Aug 7 22:00:36.611856203 (5h33m50.198912444s) [upload=5716]: Done starting upload Aug 7 22:00:36.611959786 (5h33m50.199016027s) [upload=5716]: Uploading segments... Aug 7 22:00:36.612003668 (5h33m50.199059918s) [upload=5716]: Got next segment. Inline: true Aug 7 22:00:36.612009668 (5h33m50.19906591s) [upload=5716]: Waiting for error group managing segments... Aug 7 22:00:36.612057077 (5h33m50.199113319s) [upload=5716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:00:36.612082536 (5h33m50.199138776s) [upload=5716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:00:36.612086222 (5h33m50.199142473s) [upload=5716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:00:36.884781084 (5h33m50.471837294s) [upload=5716]: Done uploading segments... Aug 7 22:00:36.884795611 (5h33m50.471851832s) [upload=5716]: Upload finished. err: <nil> Aug 7 22:01:47.555449092 (5h35m1.142505313s) [upload=5736]: Starting upload Aug 7 22:01:47.555491781 (5h35m1.142548002s) [upload=5736]: Done starting upload Aug 7 22:01:47.555562123 (5h35m1.142618344s) [upload=5736]: Uploading segments... Aug 7 22:01:47.555595826 (5h35m1.142652047s) [upload=5736]: Got next segment. Inline: true Aug 7 22:01:47.555599363 (5h35m1.142655584s) [upload=5736]: Waiting for error group managing segments... Aug 7 22:01:47.555631602 (5h35m1.142687823s) [upload=5736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:47.555635390 (5h35m1.142691611s) [upload=5736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:47.555637894 (5h35m1.142694115s) [upload=5736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:47.686110188 (5h35m1.273166429s) [upload=5736]: Done uploading segments... Aug 7 22:01:47.686151165 (5h35m1.273207406s) [upload=5736]: Upload finished. err: <nil> Aug 7 22:02:22.678566055 (5h35m36.265622267s) [upload=5746]: Starting upload Aug 7 22:02:22.678633091 (5h35m36.265689303s) [upload=5746]: Done starting upload Aug 7 22:02:22.678684407 (5h35m36.265740617s) [upload=5746]: Uploading segments... Aug 7 22:02:22.678721757 (5h35m36.265777967s) [upload=5746]: Got next segment. Inline: true Aug 7 22:02:22.678725724 (5h35m36.265781935s) [upload=5746]: Waiting for error group managing segments... Aug 7 22:02:22.678759457 (5h35m36.265815668s) [upload=5746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:02:22.678764947 (5h35m36.265821168s) [upload=5746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:02:22.678768565 (5h35m36.265824826s) [upload=5746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:02:22.971349697 (5h35m36.558405918s) [upload=5746]: Done uploading segments... Aug 7 22:02:22.971373742 (5h35m36.558429963s) [upload=5746]: Upload finished. err: <nil> Aug 7 22:03:08.907032448 (5h36m22.494088689s) [upload=5757]: Starting upload Aug 7 22:03:08.907091528 (5h36m22.494147769s) [upload=5757]: Done starting upload Aug 7 22:03:08.907172368 (5h36m22.4942286s) [upload=5757]: Uploading segments... Aug 7 22:03:08.907209458 (5h36m22.494265699s) [upload=5757]: Got next segment. Inline: true Aug 7 22:03:08.907214758 (5h36m22.494271s) [upload=5757]: Waiting for error group managing segments... Aug 7 22:03:08.907255936 (5h36m22.494312176s) [upload=5757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:03:08.907261175 (5h36m22.494317416s) [upload=5757]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:03:08.907265904 (5h36m22.494322135s) [upload=5757]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:03:09.004207779 (5h36m22.59126404s) [upload=5757]: Done uploading segments... Aug 7 22:03:09.004232375 (5h36m22.591288627s) [upload=5757]: Upload finished. err: <nil> Aug 7 22:04:02.463086309 (5h37m16.050142521s) [upload=5775]: Starting upload Aug 7 22:04:02.463128347 (5h37m16.050184568s) [upload=5775]: Done starting upload Aug 7 22:04:02.463197818 (5h37m16.050254029s) [upload=5775]: Uploading segments... Aug 7 22:04:02.463223595 (5h37m16.050279816s) [upload=5775]: Got next segment. Inline: true Aug 7 22:04:02.463227652 (5h37m16.050283873s) [upload=5775]: Waiting for error group managing segments... Aug 7 22:04:02.463258120 (5h37m16.050314331s) [upload=5775]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:04:02.463261796 (5h37m16.050318008s) [upload=5775]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:04:02.463264733 (5h37m16.050320954s) [upload=5775]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:04:02.530547234 (5h37m16.117603445s) [upload=5775]: Done uploading segments... Aug 7 22:04:02.530562512 (5h37m16.117618733s) [upload=5775]: Upload finished. err: <nil> Aug 7 22:05:10.744065032 (5h38m24.331121272s) [upload=5796]: Starting upload Aug 7 22:05:10.744195716 (5h38m24.331251957s) [upload=5796]: Done starting upload Aug 7 22:05:10.744327762 (5h38m24.331384002s) [upload=5796]: Uploading segments... Aug 7 22:05:10.744369549 (5h38m24.331425791s) [upload=5796]: Got next segment. Inline: true Aug 7 22:05:10.744388655 (5h38m24.331444897s) [upload=5796]: Waiting for error group managing segments... Aug 7 22:05:10.744449028 (5h38m24.331505259s) [upload=5796]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:05:10.744454398 (5h38m24.331510629s) [upload=5796]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:05:10.744457633 (5h38m24.331513875s) [upload=5796]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:05:10.852991578 (5h38m24.440047789s) [upload=5796]: Done uploading segments... Aug 7 22:05:10.853004571 (5h38m24.440060782s) [upload=5796]: Upload finished. err: <nil> Aug 7 22:05:45.060720865 (5h38m58.647777086s) [upload=5806]: Starting upload Aug 7 22:05:45.060780046 (5h38m58.647836267s) [upload=5806]: Done starting upload Aug 7 22:05:45.060842121 (5h38m58.647898333s) [upload=5806]: Uploading segments... Aug 7 22:05:45.060876836 (5h38m58.647933057s) [upload=5806]: Got next segment. Inline: true Aug 7 22:05:45.060882878 (5h38m58.647939099s) [upload=5806]: Waiting for error group managing segments... Aug 7 22:05:45.060927901 (5h38m58.647984112s) [upload=5806]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:05:45.060933843 (5h38m58.647990064s) [upload=5806]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:05:45.060936187 (5h38m58.647992408s) [upload=5806]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:05:45.328399453 (5h38m58.915455674s) [upload=5806]: Done uploading segments... Aug 7 22:05:45.328433367 (5h38m58.915489588s) [upload=5806]: Upload finished. err: <nil> Aug 7 22:06:12.931860875 (5h39m26.518917096s) [upload=5815]: Starting upload Aug 7 22:06:12.931905979 (5h39m26.51896219s) [upload=5815]: Done starting upload Aug 7 22:06:12.931964518 (5h39m26.51902073s) [upload=5815]: Uploading segments... Aug 7 22:06:12.931990617 (5h39m26.519046827s) [upload=5815]: Got next segment. Inline: true Aug 7 22:06:12.931994224 (5h39m26.519050434s) [upload=5815]: Waiting for error group managing segments... Aug 7 22:06:12.932024821 (5h39m26.519081042s) [upload=5815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:06:12.932028408 (5h39m26.519084618s) [upload=5815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:06:12.932030893 (5h39m26.519087103s) [upload=5815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:06:13.014995076 (5h39m26.602051317s) [upload=5815]: Done uploading segments... Aug 7 22:06:13.015014863 (5h39m26.602071114s) [upload=5815]: Upload finished. err: <nil> Aug 7 22:07:16.916734820 (5h40m30.503791062s) [upload=5832]: Starting upload Aug 7 22:07:16.916800323 (5h40m30.503856563s) [upload=5832]: Done starting upload Aug 7 22:07:16.916864883 (5h40m30.503921123s) [upload=5832]: Uploading segments... Aug 7 22:07:16.916903304 (5h40m30.503959546s) [upload=5832]: Got next segment. Inline: true Aug 7 22:07:16.916908815 (5h40m30.503965056s) [upload=5832]: Waiting for error group managing segments... Aug 7 22:07:16.916952356 (5h40m30.504008598s) [upload=5832]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:07:16.916957657 (5h40m30.504013897s) [upload=5832]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:07:16.916961754 (5h40m30.504017995s) [upload=5832]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:07:16.988776316 (5h40m30.575832566s) [upload=5832]: Done uploading segments... Aug 7 22:07:16.988801573 (5h40m30.575857824s) [upload=5832]: Upload finished. err: <nil> Aug 7 22:08:29.650294124 (5h41m43.237350365s) [upload=5854]: Starting upload Aug 7 22:08:29.650357383 (5h41m43.237413623s) [upload=5854]: Done starting upload Aug 7 22:08:29.650418325 (5h41m43.237474567s) [upload=5854]: Uploading segments... Aug 7 22:08:29.650468490 (5h41m43.237524731s) [upload=5854]: Got next segment. Inline: true Aug 7 22:08:29.650474350 (5h41m43.237530592s) [upload=5854]: Waiting for error group managing segments... Aug 7 22:08:29.650515618 (5h41m43.237571859s) [upload=5854]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:08:29.650522029 (5h41m43.237578271s) [upload=5854]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:08:29.650525556 (5h41m43.237581797s) [upload=5854]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:08:29.706715553 (5h41m43.293771793s) [upload=5854]: Done uploading segments... Aug 7 22:08:29.706731001 (5h41m43.293787243s) [upload=5854]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.501416586s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)