Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..................................................XXXXXXXXXX] duration: 30.001176671s FAIL (2025-08-07 21:24:30 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.001196216s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 13 out of 60 is failed (threshold=3) [...............................................XXXXXXXXXXXXX] duration: 30.000544178s FAIL (2025-08-07 21:24:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000563755s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 437.789949ms FAIL (2025-08-07 21:24:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 437.811408ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [................................................XXXXXXXXXXXX] duration: 30.00060655s FAIL (2025-08-07 21:24:20 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 2JRFAYC7KLC3TIXQ4PJZTKZLY552JJMZF6XTCFMHFD3RGIUMOCQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.270112598 (4h52m7.857168849s) [noise=true , address=45.91.116.8:4048 , piece_id=2IZI3WOTOBBKYHKVPJDQ6PGS5VTDAFIGTI373VWKT7QLAZR7W3FA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.270947997 (4h52m7.858004239s) [noise=true , address=45.91.116.5:4297 , piece_id=YK7QNB23S43MPEVNDLAT5WBFAMVBPQ3CVYKEMW5VBUHUAI5W4S5Q, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.271046852 (4h52m7.858103094s) [noise=true , address=77.90.31.13:4628 , piece_id=N3UNNLVQJ3GALE77VUERGE4PJDIYJK4T5VJEARML2QSMNJ2AAPJA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.272223499 (4h52m7.85927975s) [noise=true , address=45.91.116.10:4083 , piece_id=HW5YZYWXC7XHQA37JIKXDA7SSO6HOVCCGDS2SHWP7ROPG6MHQ7ZA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.272541713 (4h52m7.859597954s) [noise=true , address=45.91.116.14:4189 , piece_id=GRDSWIHHZ4FEWBEHRY7WPEF546PJWGBIXXCYDESVV4N3GSAWVGYA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.273006260 (4h52m7.860062501s) [noise=true , address=45.91.116.8:4047 , piece_id=7F4MB4X3CJZH3DUXC7VXN3HGQBBRFUV62G4KLCKV7BSARABJENWQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.275575946 (4h52m7.862632188s) [noise=true , address=77.90.31.10:4520 , piece_id=Z5P2H3YFJDC5WMKMFIFVYGFVAU2R27KTNMIAYBDYZGRUT5TWN5QQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.281367938 (4h52m7.868424169s) [noise=true , address=77.90.31.15:4697 , piece_id=PILNZYYWJABT57ITSG77YXQZN7EJKSDHB75RJHE6SPB67FSWCPGQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.284511737 (4h52m7.871567998s) [noise=true , address=77.90.31.19:4841 , piece_id=AUK7XFVC5R73KNHLVVJVRGUTRWBEJ6LP4FZIUJ77DECJRXMHWRRA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.291770877 (4h52m7.878827127s) [noise=true , address=104.36.82.4:4803 , piece_id=DCMHD25G5K5S33FQ72K7VEQ3U6ZV5UBZK5UFVDWUI5I3BX6E4NHA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.298214294 (4h52m7.885270545s) [noise=true , address=77.90.31.11:4555 , piece_id=D7YHBQ32VTIJEC4HKWQYHWMOULL3KNN7P7TJSA7LV4TLUICYGWBQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.532849875 (4h52m8.119906117s) [noise=true , address=104.36.82.6:4876 , piece_id=6WRCDS47O3FUJLE2BT6QV3OOXKGONXLHZ7TMPXKHH4LP2AJGHV5Q, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.532867749 (4h52m8.11992399s) [seg_pos={0 0}, upload=5025]: Segment reached optimal threshold of 13 pieces. Aug 7 21:18:54.533278306 (4h52m8.120334548s) [noise=true , address=104.36.82.15:4259 , piece_id=ZNJ3QCHTB2LJVM7KI52E3RPGDYEHWWFTEEBBNUIOVPIDOQP7VTKQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Aug 7 21:18:54.533410502 (4h52m8.120466743s) [noise=true , address=104.36.82.15:4258 , piece_id=VT3VKJFUHTSDCNR7PIBIJXEDYG3HGIVT4JZRR5VM3WJ6A5RHQQZQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: expected piece hash; context canceled; EOF Aug 7 21:18:54.533558579 (4h52m8.12061482s) [noise=true , address=104.36.82.14:4223 , piece_id=LX33NZFO2TFV43ARCAIGFYXJZZRR633JWZTHSO6XWSRP7TQXU5HQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: write tcp 5.161.89.252:47372->104.36.82.14:4223: use of closed network connection; write tcp 5.161.89.252:47372->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47372->104.36.82.14:4223: use of closed network connection Aug 7 21:18:54.533575210 (4h52m8.120631451s) [seg_pos={0 0}, upload=5025]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:18:54.533666891 (4h52m8.120723132s) [upload=5025]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:18:54.533672582 (4h52m8.120728823s) [upload=5025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:20:54.197308723 (4h54m7.784364965s) [upload=5025]: Done uploading segments... Aug 7 21:20:54.197331416 (4h54m7.784387657s) [upload=5025]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:54.289617342 (4h54m7.876673583s) [upload=5062]: Starting upload Aug 7 21:20:54.289682233 (4h54m7.876738475s) [upload=5062]: Done starting upload Aug 7 21:20:54.290100865 (4h54m7.877157115s) [upload=5062]: Uploading segments... Aug 7 21:20:54.290145758 (4h54m7.877201999s) [upload=5062]: Got next segment. Inline: false Aug 7 21:20:54.290178920 (4h54m7.877235161s) [upload=5062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:54.290184501 (4h54m7.877240741s) [upload=5062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:21:24.274797304 (4h54m37.861853546s) [upload=5062]: Next segment err: context deadline exceeded Aug 7 21:21:24.274996005 (4h54m37.862052246s) [upload=5062]: Done uploading segments... Aug 7 21:21:24.275019569 (4h54m37.86207581s) [upload=5062]: Upload finished. err: context deadline exceeded Aug 7 21:21:31.936040213 (4h54m45.523096464s) [upload=5073]: Starting upload Aug 7 21:21:31.936133387 (4h54m45.523189638s) [upload=5073]: Done starting upload Aug 7 21:21:31.936256627 (4h54m45.523312868s) [upload=5073]: Uploading segments... Aug 7 21:21:31.936495583 (4h54m45.523551824s) [upload=5073]: Got next segment. Inline: false Aug 7 21:21:31.936532813 (4h54m45.523589053s) [upload=5073]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:31.936537992 (4h54m45.523594234s) [upload=5073]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:22:01.921915749 (4h55m15.50897196s) [upload=5073]: Next segment err: context deadline exceeded Aug 7 21:22:01.922047263 (4h55m15.509103475s) [upload=5073]: Done uploading segments... Aug 7 21:22:01.922079755 (4h55m15.509135976s) [upload=5073]: Upload finished. err: context deadline exceeded Aug 7 21:22:02.000538540 (4h55m15.587594761s) [upload=5082]: Starting upload Aug 7 21:22:02.000624420 (4h55m15.587680652s) [upload=5082]: Done starting upload Aug 7 21:22:02.000735858 (4h55m15.587792089s) [upload=5082]: Uploading segments... Aug 7 21:22:02.000881511 (4h55m15.587937761s) [upload=5082]: Got next segment. Inline: false Aug 7 21:22:02.000947453 (4h55m15.588003684s) [upload=5082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:02.000954897 (4h55m15.588011118s) [upload=5082]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:22:31.979343661 (4h55m45.566399882s) [upload=5082]: Next segment err: context deadline exceeded Aug 7 21:22:31.979438208 (4h55m45.566494418s) [upload=5082]: Done uploading segments... Aug 7 21:22:31.979474655 (4h55m45.566530877s) [upload=5082]: Upload finished. err: context deadline exceeded Aug 7 21:22:41.968187982 (4h55m55.555244194s) [upload=5089]: Starting upload Aug 7 21:22:41.968245641 (4h55m55.555301862s) [upload=5089]: Done starting upload Aug 7 21:22:41.968339846 (4h55m55.555396087s) [upload=5089]: Uploading segments... Aug 7 21:22:41.968596806 (4h55m55.555653047s) [upload=5089]: Got next segment. Inline: false Aug 7 21:22:41.968638122 (4h55m55.555694354s) [upload=5089]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:41.968644665 (4h55m55.555700906s) [upload=5089]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:23:11.943981705 (4h56m25.531037956s) [upload=5089]: Next segment err: context deadline exceeded Aug 7 21:23:11.944159476 (4h56m25.531215718s) [upload=5089]: Done uploading segments... Aug 7 21:23:11.944182590 (4h56m25.53123883s) [upload=5089]: Upload finished. err: context deadline exceeded Aug 7 21:23:20.066472018 (4h56m33.653528259s) [upload=5100]: Starting upload Aug 7 21:23:20.066538723 (4h56m33.653594965s) [upload=5100]: Done starting upload Aug 7 21:23:20.066650672 (4h56m33.653706913s) [upload=5100]: Uploading segments... Aug 7 21:23:20.066882696 (4h56m33.653938936s) [upload=5100]: Got next segment. Inline: false Aug 7 21:23:20.066929522 (4h56m33.653985764s) [upload=5100]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:20.066934632 (4h56m33.653990873s) [upload=5100]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:23:50.045530632 (4h57m3.632586874s) [upload=5100]: Next segment err: context deadline exceeded Aug 7 21:23:50.045695360 (4h57m3.6327516s) [upload=5100]: Done uploading segments... Aug 7 21:23:50.045720867 (4h57m3.632777109s) [upload=5100]: Upload finished. err: context deadline exceeded Aug 7 21:23:50.132503139 (4h57m3.719559361s) [upload=5111]: Starting upload Aug 7 21:23:50.132542101 (4h57m3.719598322s) [upload=5111]: Done starting upload Aug 7 21:23:50.132574512 (4h57m3.719630733s) [upload=5111]: Uploading segments... Aug 7 21:23:50.132772312 (4h57m3.719828533s) [upload=5111]: Got next segment. Inline: false Aug 7 21:23:50.132810694 (4h57m3.719866904s) [upload=5111]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:50.132814390 (4h57m3.719870611s) [upload=5111]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:24:20.119434924 (4h57m33.706491135s) [upload=5111]: Next segment err: context deadline exceeded Aug 7 21:24:20.119485478 (4h57m33.706541699s) [upload=5111]: Done uploading segments... Aug 7 21:24:20.119497891 (4h57m33.706554102s) [upload=5111]: Upload finished. err: context deadline exceeded Duration: 30.000616449s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [...............................................XXXXXXXXXXXXX] duration: 30.001209675s FAIL (2025-08-07 21:24:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001222539s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 15 out of 60 is failed (threshold=3) [.............................................XXXXXXXXXXXXXXX] duration: 30.001766085s FAIL (2025-08-07 21:24:14 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d Aug 7 21:16:20.227399180 (4h49m33.814455401s) [upload=4990]: Starting upload Aug 7 21:16:20.227450334 (4h49m33.814506545s) [upload=4990]: Done starting upload Aug 7 21:16:20.227491060 (4h49m33.814547272s) [upload=4990]: Uploading segments... Aug 7 21:16:20.227516609 (4h49m33.81457283s) [upload=4990]: Got next segment. Inline: true Aug 7 21:16:20.227520536 (4h49m33.814576757s) [upload=4990]: Waiting for error group managing segments... Aug 7 21:16:20.227552426 (4h49m33.814608636s) [upload=4990]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:16:20.227556263 (4h49m33.814612484s) [upload=4990]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:20.227558527 (4h49m33.814614739s) [upload=4990]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:16:50.216081431 (4h50m3.803137652s) [upload=4990]: Done uploading segments... Aug 7 21:16:50.216097492 (4h50m3.803153713s) [upload=4990]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:57.322533914 (4h50m10.909590156s) [upload=4995]: Starting upload Aug 7 21:16:57.322609355 (4h50m10.909665607s) [upload=4995]: Done starting upload Aug 7 21:16:57.322672883 (4h50m10.909729125s) [upload=4995]: Uploading segments... Aug 7 21:16:57.322711135 (4h50m10.909767376s) [upload=4995]: Got next segment. Inline: true Aug 7 21:16:57.322716685 (4h50m10.909772927s) [upload=4995]: Waiting for error group managing segments... Aug 7 21:16:57.322761279 (4h50m10.90981752s) [upload=4995]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:16:57.322767100 (4h50m10.909823341s) [upload=4995]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:57.322770697 (4h50m10.909826938s) [upload=4995]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:17:27.310589172 (4h50m40.897645383s) [upload=4995]: Done uploading segments... Aug 7 21:17:27.310603289 (4h50m40.897659499s) [upload=4995]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:38.458031408 (4h50m52.045087639s) [upload=5010]: Starting upload Aug 7 21:17:38.458135221 (4h50m52.045191452s) [upload=5010]: Done starting upload Aug 7 21:17:38.458199552 (4h50m52.045255784s) [upload=5010]: Uploading segments... Aug 7 21:17:38.458275263 (4h50m52.045331504s) [upload=5010]: Got next segment. Inline: true Aug 7 21:17:38.458283519 (4h50m52.04533976s) [upload=5010]: Waiting for error group managing segments... Aug 7 21:17:38.458329695 (4h50m52.045385926s) [upload=5010]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:17:38.458335165 (4h50m52.045391407s) [upload=5010]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:38.458338662 (4h50m52.045394893s) [upload=5010]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:18:08.444937554 (4h51m22.031993765s) [upload=5010]: Done uploading segments... Aug 7 21:18:08.444950428 (4h51m22.032006639s) [upload=5010]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:50.205377240 (4h52m3.792433451s) [upload=5028]: Starting upload Aug 7 21:18:50.205432072 (4h52m3.792488293s) [upload=5028]: Done starting upload Aug 7 21:18:50.205472938 (4h52m3.792529149s) [upload=5028]: Uploading segments... Aug 7 21:18:50.205500099 (4h52m3.792556311s) [upload=5028]: Got next segment. Inline: true Aug 7 21:18:50.205503937 (4h52m3.792560147s) [upload=5028]: Waiting for error group managing segments... Aug 7 21:18:50.205531809 (4h52m3.79258802s) [upload=5028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:18:50.205535356 (4h52m3.792591577s) [upload=5028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:50.205538692 (4h52m3.792594913s) [upload=5028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:19:20.193730995 (4h52m33.780787246s) [upload=5028]: Done uploading segments... Aug 7 21:19:20.193750121 (4h52m33.780806372s) [upload=5028]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:03.998495850 (4h53m17.58555209s) [upload=5048]: Starting upload Aug 7 21:20:03.998559819 (4h53m17.585616059s) [upload=5048]: Done starting upload Aug 7 21:20:03.998621224 (4h53m17.585677465s) [upload=5048]: Uploading segments... Aug 7 21:20:03.998661639 (4h53m17.585717879s) [upload=5048]: Got next segment. Inline: true Aug 7 21:20:03.998667419 (4h53m17.58572365s) [upload=5048]: Waiting for error group managing segments... Aug 7 21:20:03.998710790 (4h53m17.585767032s) [upload=5048]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:03.998716000 (4h53m17.585772242s) [upload=5048]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:20:03.998719587 (4h53m17.585775818s) [upload=5048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:20:33.984947930 (4h53m47.572004141s) [upload=5048]: Done uploading segments... Aug 7 21:20:33.985025244 (4h53m47.572081455s) [upload=5048]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:03.692382456 (4h54m17.279438677s) [upload=5064]: Starting upload Aug 7 21:21:03.692428231 (4h54m17.279484453s) [upload=5064]: Done starting upload Aug 7 21:21:03.692467736 (4h54m17.279523957s) [upload=5064]: Uploading segments... Aug 7 21:21:03.692495146 (4h54m17.279551367s) [upload=5064]: Got next segment. Inline: true Aug 7 21:21:03.692500066 (4h54m17.279556287s) [upload=5064]: Waiting for error group managing segments... Aug 7 21:21:03.692529661 (4h54m17.279585882s) [upload=5064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:03.692533137 (4h54m17.279589349s) [upload=5064]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:03.692535481 (4h54m17.279591702s) [upload=5064]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:21:33.679184065 (4h54m47.266240287s) [upload=5064]: Done uploading segments... Aug 7 21:21:33.679200356 (4h54m47.266256568s) [upload=5064]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:41.028727083 (4h54m54.615783293s) [upload=5077]: Starting upload Aug 7 21:21:41.028776044 (4h54m54.615832265s) [upload=5077]: Done starting upload Aug 7 21:21:41.028822871 (4h54m54.615879093s) [upload=5077]: Uploading segments... Aug 7 21:21:41.028849481 (4h54m54.615905693s) [upload=5077]: Got next segment. Inline: true Aug 7 21:21:41.028853158 (4h54m54.61590938s) [upload=5077]: Waiting for error group managing segments... Aug 7 21:21:41.028887172 (4h54m54.615943383s) [upload=5077]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:41.028891160 (4h54m54.615947381s) [upload=5077]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:41.028893564 (4h54m54.615949785s) [upload=5077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:11.020606617 (4h55m24.607662858s) [upload=5077]: Done uploading segments... Aug 7 21:22:11.020629420 (4h55m24.60768566s) [upload=5077]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:22:44.704346518 (4h55m58.291402739s) [upload=5092]: Starting upload Aug 7 21:22:44.704395569 (4h55m58.29145179s) [upload=5092]: Done starting upload Aug 7 21:22:44.704432468 (4h55m58.291488689s) [upload=5092]: Uploading segments... Aug 7 21:22:44.704457685 (4h55m58.291513906s) [upload=5092]: Got next segment. Inline: true Aug 7 21:22:44.704461182 (4h55m58.291517403s) [upload=5092]: Waiting for error group managing segments... Aug 7 21:22:44.704487161 (4h55m58.291543382s) [upload=5092]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:44.704490647 (4h55m58.291546868s) [upload=5092]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:44.704493612 (4h55m58.291549833s) [upload=5092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:14.694775680 (4h56m28.281831901s) [upload=5092]: Done uploading segments... Aug 7 21:23:14.694790848 (4h56m28.281847069s) [upload=5092]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:14.781991861 (4h56m28.369048072s) [upload=5099]: Starting upload Aug 7 21:23:14.782035392 (4h56m28.369091604s) [upload=5099]: Done starting upload Aug 7 21:23:14.782087559 (4h56m28.369143771s) [upload=5099]: Uploading segments... Aug 7 21:23:14.782124759 (4h56m28.369180991s) [upload=5099]: Got next segment. Inline: true Aug 7 21:23:14.782130029 (4h56m28.369186241s) [upload=5099]: Waiting for error group managing segments... Aug 7 21:23:14.782161368 (4h56m28.369217579s) [upload=5099]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:14.782164914 (4h56m28.369221126s) [upload=5099]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:14.782167850 (4h56m28.369224061s) [upload=5099]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:44.771505754 (4h56m58.358561975s) [upload=5099]: Done uploading segments... Aug 7 21:23:44.771536371 (4h56m58.358592583s) [upload=5099]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:44.852200021 (4h56m58.439256242s) [upload=5107]: Starting upload Aug 7 21:23:44.852251448 (4h56m58.439307669s) [upload=5107]: Done starting upload Aug 7 21:23:44.852291703 (4h56m58.439347924s) [upload=5107]: Uploading segments... Aug 7 21:23:44.852320757 (4h56m58.439376978s) [upload=5107]: Got next segment. Inline: true Aug 7 21:23:44.852324224 (4h56m58.439380445s) [upload=5107]: Waiting for error group managing segments... Aug 7 21:23:44.852355542 (4h56m58.439411753s) [upload=5107]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:44.852359279 (4h56m58.4394155s) [upload=5107]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:44.852361614 (4h56m58.439417824s) [upload=5107]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:14.835050939 (4h57m28.42210715s) [upload=5107]: Done uploading segments... Aug 7 21:24:14.835115480 (4h57m28.422171701s) [upload=5107]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.001778269s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 11 out of 60 is failed (threshold=3) [................................................XX.XXXXXXXXX] duration: 1m0.001019979s FAIL (2025-08-07 21:23:58 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001059673s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 12 out of 60 is failed (threshold=3) [................................................XXXXXXXXXXXX] duration: 30.000959454s FAIL (2025-08-07 21:23:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000978921s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 15 out of 60 is failed (threshold=3) [...........................................XX..XXXXXXXXXXXXX] duration: 30.00117655s FAIL (2025-08-07 21:23:51 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.00119823s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.................................................XXXXXXXXXXX] duration: 30.001417749s FAIL (2025-08-07 21:23:50 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 09878973s) [noise=true , address=77.90.31.13:4628 , piece_id=N3UNNLVQJ3GALE77VUERGE4PJDIYJK4T5VJEARML2QSMNJ2AAPJA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.123209864 (4h52m7.710266105s) [noise=true , address=45.91.116.5:4297 , piece_id=YK7QNB23S43MPEVNDLAT5WBFAMVBPQ3CVYKEMW5VBUHUAI5W4S5Q, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.123571199 (4h52m7.710627439s) [noise=true , address=104.36.82.14:4223 , piece_id=LX33NZFO2TFV43ARCAIGFYXJZZRR633JWZTHSO6XWSRP7TQXU5HQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.259677990 (4h52m7.846734232s) [upload=5025]: Next returned nil segment Aug 7 21:18:54.259835595 (4h52m7.846891847s) [upload=5025]: Waiting for error group managing segments... Aug 7 21:18:54.269953672 (4h52m7.857009913s) [noise=true , address=45.91.116.7:4011 , piece_id=H2JRFAYC7KLC3TIXQ4PJZTKZLY552JJMZF6XTCFMHFD3RGIUMOCQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.270112598 (4h52m7.857168849s) [noise=true , address=45.91.116.8:4048 , piece_id=2IZI3WOTOBBKYHKVPJDQ6PGS5VTDAFIGTI373VWKT7QLAZR7W3FA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.270947997 (4h52m7.858004239s) [noise=true , address=45.91.116.5:4297 , piece_id=YK7QNB23S43MPEVNDLAT5WBFAMVBPQ3CVYKEMW5VBUHUAI5W4S5Q, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.271046852 (4h52m7.858103094s) [noise=true , address=77.90.31.13:4628 , piece_id=N3UNNLVQJ3GALE77VUERGE4PJDIYJK4T5VJEARML2QSMNJ2AAPJA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.272223499 (4h52m7.85927975s) [noise=true , address=45.91.116.10:4083 , piece_id=HW5YZYWXC7XHQA37JIKXDA7SSO6HOVCCGDS2SHWP7ROPG6MHQ7ZA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.272541713 (4h52m7.859597954s) [noise=true , address=45.91.116.14:4189 , piece_id=GRDSWIHHZ4FEWBEHRY7WPEF546PJWGBIXXCYDESVV4N3GSAWVGYA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.273006260 (4h52m7.860062501s) [noise=true , address=45.91.116.8:4047 , piece_id=7F4MB4X3CJZH3DUXC7VXN3HGQBBRFUV62G4KLCKV7BSARABJENWQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.275575946 (4h52m7.862632188s) [noise=true , address=77.90.31.10:4520 , piece_id=Z5P2H3YFJDC5WMKMFIFVYGFVAU2R27KTNMIAYBDYZGRUT5TWN5QQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.281367938 (4h52m7.868424169s) [noise=true , address=77.90.31.15:4697 , piece_id=PILNZYYWJABT57ITSG77YXQZN7EJKSDHB75RJHE6SPB67FSWCPGQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.284511737 (4h52m7.871567998s) [noise=true , address=77.90.31.19:4841 , piece_id=AUK7XFVC5R73KNHLVVJVRGUTRWBEJ6LP4FZIUJ77DECJRXMHWRRA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.291770877 (4h52m7.878827127s) [noise=true , address=104.36.82.4:4803 , piece_id=DCMHD25G5K5S33FQ72K7VEQ3U6ZV5UBZK5UFVDWUI5I3BX6E4NHA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.298214294 (4h52m7.885270545s) [noise=true , address=77.90.31.11:4555 , piece_id=D7YHBQ32VTIJEC4HKWQYHWMOULL3KNN7P7TJSA7LV4TLUICYGWBQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.532849875 (4h52m8.119906117s) [noise=true , address=104.36.82.6:4876 , piece_id=6WRCDS47O3FUJLE2BT6QV3OOXKGONXLHZ7TMPXKHH4LP2AJGHV5Q, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.532867749 (4h52m8.11992399s) [seg_pos={0 0}, upload=5025]: Segment reached optimal threshold of 13 pieces. Aug 7 21:18:54.533278306 (4h52m8.120334548s) [noise=true , address=104.36.82.15:4259 , piece_id=ZNJ3QCHTB2LJVM7KI52E3RPGDYEHWWFTEEBBNUIOVPIDOQP7VTKQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Aug 7 21:18:54.533410502 (4h52m8.120466743s) [noise=true , address=104.36.82.15:4258 , piece_id=VT3VKJFUHTSDCNR7PIBIJXEDYG3HGIVT4JZRR5VM3WJ6A5RHQQZQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: expected piece hash; context canceled; EOF Aug 7 21:18:54.533558579 (4h52m8.12061482s) [noise=true , address=104.36.82.14:4223 , piece_id=LX33NZFO2TFV43ARCAIGFYXJZZRR633JWZTHSO6XWSRP7TQXU5HQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: write tcp 5.161.89.252:47372->104.36.82.14:4223: use of closed network connection; write tcp 5.161.89.252:47372->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47372->104.36.82.14:4223: use of closed network connection Aug 7 21:18:54.533575210 (4h52m8.120631451s) [seg_pos={0 0}, upload=5025]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:18:54.533666891 (4h52m8.120723132s) [upload=5025]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:18:54.533672582 (4h52m8.120728823s) [upload=5025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:20:54.197308723 (4h54m7.784364965s) [upload=5025]: Done uploading segments... Aug 7 21:20:54.197331416 (4h54m7.784387657s) [upload=5025]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:54.289617342 (4h54m7.876673583s) [upload=5062]: Starting upload Aug 7 21:20:54.289682233 (4h54m7.876738475s) [upload=5062]: Done starting upload Aug 7 21:20:54.290100865 (4h54m7.877157115s) [upload=5062]: Uploading segments... Aug 7 21:20:54.290145758 (4h54m7.877201999s) [upload=5062]: Got next segment. Inline: false Aug 7 21:20:54.290178920 (4h54m7.877235161s) [upload=5062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:54.290184501 (4h54m7.877240741s) [upload=5062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:21:24.274797304 (4h54m37.861853546s) [upload=5062]: Next segment err: context deadline exceeded Aug 7 21:21:24.274996005 (4h54m37.862052246s) [upload=5062]: Done uploading segments... Aug 7 21:21:24.275019569 (4h54m37.86207581s) [upload=5062]: Upload finished. err: context deadline exceeded Aug 7 21:21:31.936040213 (4h54m45.523096464s) [upload=5073]: Starting upload Aug 7 21:21:31.936133387 (4h54m45.523189638s) [upload=5073]: Done starting upload Aug 7 21:21:31.936256627 (4h54m45.523312868s) [upload=5073]: Uploading segments... Aug 7 21:21:31.936495583 (4h54m45.523551824s) [upload=5073]: Got next segment. Inline: false Aug 7 21:21:31.936532813 (4h54m45.523589053s) [upload=5073]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:31.936537992 (4h54m45.523594234s) [upload=5073]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:22:01.921915749 (4h55m15.50897196s) [upload=5073]: Next segment err: context deadline exceeded Aug 7 21:22:01.922047263 (4h55m15.509103475s) [upload=5073]: Done uploading segments... Aug 7 21:22:01.922079755 (4h55m15.509135976s) [upload=5073]: Upload finished. err: context deadline exceeded Aug 7 21:22:02.000538540 (4h55m15.587594761s) [upload=5082]: Starting upload Aug 7 21:22:02.000624420 (4h55m15.587680652s) [upload=5082]: Done starting upload Aug 7 21:22:02.000735858 (4h55m15.587792089s) [upload=5082]: Uploading segments... Aug 7 21:22:02.000881511 (4h55m15.587937761s) [upload=5082]: Got next segment. Inline: false Aug 7 21:22:02.000947453 (4h55m15.588003684s) [upload=5082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:02.000954897 (4h55m15.588011118s) [upload=5082]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:22:31.979343661 (4h55m45.566399882s) [upload=5082]: Next segment err: context deadline exceeded Aug 7 21:22:31.979438208 (4h55m45.566494418s) [upload=5082]: Done uploading segments... Aug 7 21:22:31.979474655 (4h55m45.566530877s) [upload=5082]: Upload finished. err: context deadline exceeded Aug 7 21:22:41.968187982 (4h55m55.555244194s) [upload=5089]: Starting upload Aug 7 21:22:41.968245641 (4h55m55.555301862s) [upload=5089]: Done starting upload Aug 7 21:22:41.968339846 (4h55m55.555396087s) [upload=5089]: Uploading segments... Aug 7 21:22:41.968596806 (4h55m55.555653047s) [upload=5089]: Got next segment. Inline: false Aug 7 21:22:41.968638122 (4h55m55.555694354s) [upload=5089]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:41.968644665 (4h55m55.555700906s) [upload=5089]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:23:11.943981705 (4h56m25.531037956s) [upload=5089]: Next segment err: context deadline exceeded Aug 7 21:23:11.944159476 (4h56m25.531215718s) [upload=5089]: Done uploading segments... Aug 7 21:23:11.944182590 (4h56m25.53123883s) [upload=5089]: Upload finished. err: context deadline exceeded Aug 7 21:23:20.066472018 (4h56m33.653528259s) [upload=5100]: Starting upload Aug 7 21:23:20.066538723 (4h56m33.653594965s) [upload=5100]: Done starting upload Aug 7 21:23:20.066650672 (4h56m33.653706913s) [upload=5100]: Uploading segments... Aug 7 21:23:20.066882696 (4h56m33.653938936s) [upload=5100]: Got next segment. Inline: false Aug 7 21:23:20.066929522 (4h56m33.653985764s) [upload=5100]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:20.066934632 (4h56m33.653990873s) [upload=5100]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:23:50.045530632 (4h57m3.632586874s) [upload=5100]: Next segment err: context deadline exceeded Aug 7 21:23:50.045695360 (4h57m3.6327516s) [upload=5100]: Done uploading segments... Aug 7 21:23:50.045720867 (4h57m3.632777109s) [upload=5100]: Upload finished. err: context deadline exceeded Duration: 30.001436544s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 1.010368468s FAIL (2025-08-07 21:23:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 1.010390189s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 14 out of 60 is failed (threshold=3) [..............................................XXXXXXXXXXXXXX] duration: 30.000700506s FAIL (2025-08-07 21:23:44 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 7 21:15:40.993035983 (4h48m54.580092194s) [upload=4978]: Starting upload Aug 7 21:15:40.993105403 (4h48m54.580161614s) [upload=4978]: Done starting upload Aug 7 21:15:40.993154163 (4h48m54.580210375s) [upload=4978]: Uploading segments... Aug 7 21:15:40.993189519 (4h48m54.580245741s) [upload=4978]: Got next segment. Inline: true Aug 7 21:15:40.993194319 (4h48m54.58025054s) [upload=4978]: Waiting for error group managing segments... Aug 7 21:15:40.993226930 (4h48m54.580283151s) [upload=4978]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:15:40.993231167 (4h48m54.580287379s) [upload=4978]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:40.993234264 (4h48m54.580290474s) [upload=4978]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:16:10.982081529 (4h49m24.569137771s) [upload=4978]: Done uploading segments... Aug 7 21:16:10.982104473 (4h49m24.569160713s) [upload=4978]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:20.227399180 (4h49m33.814455401s) [upload=4990]: Starting upload Aug 7 21:16:20.227450334 (4h49m33.814506545s) [upload=4990]: Done starting upload Aug 7 21:16:20.227491060 (4h49m33.814547272s) [upload=4990]: Uploading segments... Aug 7 21:16:20.227516609 (4h49m33.81457283s) [upload=4990]: Got next segment. Inline: true Aug 7 21:16:20.227520536 (4h49m33.814576757s) [upload=4990]: Waiting for error group managing segments... Aug 7 21:16:20.227552426 (4h49m33.814608636s) [upload=4990]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:16:20.227556263 (4h49m33.814612484s) [upload=4990]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:20.227558527 (4h49m33.814614739s) [upload=4990]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:16:50.216081431 (4h50m3.803137652s) [upload=4990]: Done uploading segments... Aug 7 21:16:50.216097492 (4h50m3.803153713s) [upload=4990]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:57.322533914 (4h50m10.909590156s) [upload=4995]: Starting upload Aug 7 21:16:57.322609355 (4h50m10.909665607s) [upload=4995]: Done starting upload Aug 7 21:16:57.322672883 (4h50m10.909729125s) [upload=4995]: Uploading segments... Aug 7 21:16:57.322711135 (4h50m10.909767376s) [upload=4995]: Got next segment. Inline: true Aug 7 21:16:57.322716685 (4h50m10.909772927s) [upload=4995]: Waiting for error group managing segments... Aug 7 21:16:57.322761279 (4h50m10.90981752s) [upload=4995]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:16:57.322767100 (4h50m10.909823341s) [upload=4995]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:57.322770697 (4h50m10.909826938s) [upload=4995]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:17:27.310589172 (4h50m40.897645383s) [upload=4995]: Done uploading segments... Aug 7 21:17:27.310603289 (4h50m40.897659499s) [upload=4995]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:38.458031408 (4h50m52.045087639s) [upload=5010]: Starting upload Aug 7 21:17:38.458135221 (4h50m52.045191452s) [upload=5010]: Done starting upload Aug 7 21:17:38.458199552 (4h50m52.045255784s) [upload=5010]: Uploading segments... Aug 7 21:17:38.458275263 (4h50m52.045331504s) [upload=5010]: Got next segment. Inline: true Aug 7 21:17:38.458283519 (4h50m52.04533976s) [upload=5010]: Waiting for error group managing segments... Aug 7 21:17:38.458329695 (4h50m52.045385926s) [upload=5010]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:17:38.458335165 (4h50m52.045391407s) [upload=5010]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:38.458338662 (4h50m52.045394893s) [upload=5010]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:18:08.444937554 (4h51m22.031993765s) [upload=5010]: Done uploading segments... Aug 7 21:18:08.444950428 (4h51m22.032006639s) [upload=5010]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:50.205377240 (4h52m3.792433451s) [upload=5028]: Starting upload Aug 7 21:18:50.205432072 (4h52m3.792488293s) [upload=5028]: Done starting upload Aug 7 21:18:50.205472938 (4h52m3.792529149s) [upload=5028]: Uploading segments... Aug 7 21:18:50.205500099 (4h52m3.792556311s) [upload=5028]: Got next segment. Inline: true Aug 7 21:18:50.205503937 (4h52m3.792560147s) [upload=5028]: Waiting for error group managing segments... Aug 7 21:18:50.205531809 (4h52m3.79258802s) [upload=5028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:18:50.205535356 (4h52m3.792591577s) [upload=5028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:50.205538692 (4h52m3.792594913s) [upload=5028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:19:20.193730995 (4h52m33.780787246s) [upload=5028]: Done uploading segments... Aug 7 21:19:20.193750121 (4h52m33.780806372s) [upload=5028]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:03.998495850 (4h53m17.58555209s) [upload=5048]: Starting upload Aug 7 21:20:03.998559819 (4h53m17.585616059s) [upload=5048]: Done starting upload Aug 7 21:20:03.998621224 (4h53m17.585677465s) [upload=5048]: Uploading segments... Aug 7 21:20:03.998661639 (4h53m17.585717879s) [upload=5048]: Got next segment. Inline: true Aug 7 21:20:03.998667419 (4h53m17.58572365s) [upload=5048]: Waiting for error group managing segments... Aug 7 21:20:03.998710790 (4h53m17.585767032s) [upload=5048]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:03.998716000 (4h53m17.585772242s) [upload=5048]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:20:03.998719587 (4h53m17.585775818s) [upload=5048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:20:33.984947930 (4h53m47.572004141s) [upload=5048]: Done uploading segments... Aug 7 21:20:33.985025244 (4h53m47.572081455s) [upload=5048]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:03.692382456 (4h54m17.279438677s) [upload=5064]: Starting upload Aug 7 21:21:03.692428231 (4h54m17.279484453s) [upload=5064]: Done starting upload Aug 7 21:21:03.692467736 (4h54m17.279523957s) [upload=5064]: Uploading segments... Aug 7 21:21:03.692495146 (4h54m17.279551367s) [upload=5064]: Got next segment. Inline: true Aug 7 21:21:03.692500066 (4h54m17.279556287s) [upload=5064]: Waiting for error group managing segments... Aug 7 21:21:03.692529661 (4h54m17.279585882s) [upload=5064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:03.692533137 (4h54m17.279589349s) [upload=5064]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:03.692535481 (4h54m17.279591702s) [upload=5064]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:21:33.679184065 (4h54m47.266240287s) [upload=5064]: Done uploading segments... Aug 7 21:21:33.679200356 (4h54m47.266256568s) [upload=5064]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:41.028727083 (4h54m54.615783293s) [upload=5077]: Starting upload Aug 7 21:21:41.028776044 (4h54m54.615832265s) [upload=5077]: Done starting upload Aug 7 21:21:41.028822871 (4h54m54.615879093s) [upload=5077]: Uploading segments... Aug 7 21:21:41.028849481 (4h54m54.615905693s) [upload=5077]: Got next segment. Inline: true Aug 7 21:21:41.028853158 (4h54m54.61590938s) [upload=5077]: Waiting for error group managing segments... Aug 7 21:21:41.028887172 (4h54m54.615943383s) [upload=5077]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:41.028891160 (4h54m54.615947381s) [upload=5077]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:41.028893564 (4h54m54.615949785s) [upload=5077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:11.020606617 (4h55m24.607662858s) [upload=5077]: Done uploading segments... Aug 7 21:22:11.020629420 (4h55m24.60768566s) [upload=5077]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:22:44.704346518 (4h55m58.291402739s) [upload=5092]: Starting upload Aug 7 21:22:44.704395569 (4h55m58.29145179s) [upload=5092]: Done starting upload Aug 7 21:22:44.704432468 (4h55m58.291488689s) [upload=5092]: Uploading segments... Aug 7 21:22:44.704457685 (4h55m58.291513906s) [upload=5092]: Got next segment. Inline: true Aug 7 21:22:44.704461182 (4h55m58.291517403s) [upload=5092]: Waiting for error group managing segments... Aug 7 21:22:44.704487161 (4h55m58.291543382s) [upload=5092]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:44.704490647 (4h55m58.291546868s) [upload=5092]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:44.704493612 (4h55m58.291549833s) [upload=5092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:14.694775680 (4h56m28.281831901s) [upload=5092]: Done uploading segments... Aug 7 21:23:14.694790848 (4h56m28.281847069s) [upload=5092]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:14.781991861 (4h56m28.369048072s) [upload=5099]: Starting upload Aug 7 21:23:14.782035392 (4h56m28.369091604s) [upload=5099]: Done starting upload Aug 7 21:23:14.782087559 (4h56m28.369143771s) [upload=5099]: Uploading segments... Aug 7 21:23:14.782124759 (4h56m28.369180991s) [upload=5099]: Got next segment. Inline: true Aug 7 21:23:14.782130029 (4h56m28.369186241s) [upload=5099]: Waiting for error group managing segments... Aug 7 21:23:14.782161368 (4h56m28.369217579s) [upload=5099]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:14.782164914 (4h56m28.369221126s) [upload=5099]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:14.782167850 (4h56m28.369224061s) [upload=5099]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:44.771505754 (4h56m58.358561975s) [upload=5099]: Done uploading segments... Aug 7 21:23:44.771536371 (4h56m58.358592583s) [upload=5099]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.000713289s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 9 out of 60 is failed (threshold=3) [...................................................XXXXXXXXX] duration: 1m0.000406881s FAIL (2025-08-07 21:23:44 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d finished. err: <nil> Aug 7 21:13:23.238906839 (4h46m36.82596306s) [upload=4942]: Starting upload Aug 7 21:13:23.238959356 (4h46m36.826015567s) [upload=4942]: Done starting upload Aug 7 21:13:23.239011013 (4h46m36.826067234s) [upload=4942]: Uploading segments... Aug 7 21:13:23.239039727 (4h46m36.826095937s) [upload=4942]: Got next segment. Inline: true Aug 7 21:13:23.239043483 (4h46m36.826099695s) [upload=4942]: Waiting for error group managing segments... Aug 7 21:13:23.239098526 (4h46m36.826154737s) [upload=4942]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:13:23.239104707 (4h46m36.826160919s) [upload=4942]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:13:23.239108474 (4h46m36.826164696s) [upload=4942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:13:23.309357825 (4h46m36.896414127s) [upload=4942]: Done uploading segments... Aug 7 21:13:23.309383974 (4h46m36.896440216s) [upload=4942]: Upload finished. err: <nil> Aug 7 21:14:18.294119361 (4h47m31.881175572s) [upload=4957]: Starting upload Aug 7 21:14:18.294187468 (4h47m31.881243679s) [upload=4957]: Done starting upload Aug 7 21:14:18.294261516 (4h47m31.881317737s) [upload=4957]: Uploading segments... Aug 7 21:14:18.294305068 (4h47m31.881361289s) [upload=4957]: Got next segment. Inline: true Aug 7 21:14:18.294311409 (4h47m31.881367621s) [upload=4957]: Waiting for error group managing segments... Aug 7 21:14:18.294349491 (4h47m31.881405712s) [upload=4957]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:18.294354059 (4h47m31.88141028s) [upload=4957]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:14:18.294356643 (4h47m31.881412854s) [upload=4957]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:15:18.280461255 (4h48m31.867517496s) [upload=4957]: Done uploading segments... Aug 7 21:15:18.280484308 (4h48m31.867540559s) [upload=4957]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:15:18.387536247 (4h48m31.974592488s) [upload=4974]: Starting upload Aug 7 21:15:18.387607770 (4h48m31.974664011s) [upload=4974]: Done starting upload Aug 7 21:15:18.387666109 (4h48m31.974722361s) [upload=4974]: Uploading segments... Aug 7 21:15:18.387706996 (4h48m31.974763236s) [upload=4974]: Got next segment. Inline: true Aug 7 21:15:18.387712766 (4h48m31.974769007s) [upload=4974]: Waiting for error group managing segments... Aug 7 21:15:18.387755796 (4h48m31.974812038s) [upload=4974]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:15:18.387761227 (4h48m31.974817468s) [upload=4974]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:18.387764853 (4h48m31.974821095s) [upload=4974]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:16:18.373970870 (4h49m31.961027121s) [upload=4974]: Done uploading segments... Aug 7 21:16:18.374035110 (4h49m31.961091352s) [upload=4974]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:18.454541239 (4h49m32.04159748s) [upload=4989]: Starting upload Aug 7 21:16:18.454621288 (4h49m32.041677529s) [upload=4989]: Done starting upload Aug 7 21:16:18.454699754 (4h49m32.041756005s) [upload=4989]: Uploading segments... Aug 7 21:16:18.454758153 (4h49m32.041814395s) [upload=4989]: Got next segment. Inline: true Aug 7 21:16:18.454765147 (4h49m32.041821398s) [upload=4989]: Waiting for error group managing segments... Aug 7 21:16:18.454813777 (4h49m32.041870018s) [upload=4989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:16:18.454819818 (4h49m32.04187606s) [upload=4989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:18.454823425 (4h49m32.041879666s) [upload=4989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:17:18.435319201 (4h50m32.022375413s) [upload=4989]: Done uploading segments... Aug 7 21:17:18.435335041 (4h50m32.022391262s) [upload=4989]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:29.913551414 (4h50m43.500607635s) [upload=5005]: Starting upload Aug 7 21:17:29.913606246 (4h50m43.500662467s) [upload=5005]: Done starting upload Aug 7 21:17:29.913653174 (4h50m43.500709395s) [upload=5005]: Uploading segments... Aug 7 21:17:29.913680986 (4h50m43.500737207s) [upload=5005]: Got next segment. Inline: true Aug 7 21:17:29.913684613 (4h50m43.500740835s) [upload=5005]: Waiting for error group managing segments... Aug 7 21:17:29.913715249 (4h50m43.500771471s) [upload=5005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:17:29.913719057 (4h50m43.500775278s) [upload=5005]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:29.913721331 (4h50m43.500777542s) [upload=5005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:18:29.900370534 (4h51m43.487426745s) [upload=5005]: Done uploading segments... Aug 7 21:18:29.900390291 (4h51m43.487446542s) [upload=5005]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:29.988006379 (4h51m43.5750626s) [upload=5023]: Starting upload Aug 7 21:18:29.988053607 (4h51m43.575109818s) [upload=5023]: Done starting upload Aug 7 21:18:29.988109782 (4h51m43.575166003s) [upload=5023]: Uploading segments... Aug 7 21:18:29.988142012 (4h51m43.575198224s) [upload=5023]: Got next segment. Inline: true Aug 7 21:18:29.988147753 (4h51m43.575203973s) [upload=5023]: Waiting for error group managing segments... Aug 7 21:18:29.988197475 (4h51m43.575253687s) [upload=5023]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:18:29.988203116 (4h51m43.575259327s) [upload=5023]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:29.988206943 (4h51m43.575263164s) [upload=5023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:19:29.967158594 (4h52m43.554214836s) [upload=5023]: Done uploading segments... Aug 7 21:19:29.967184642 (4h52m43.554240884s) [upload=5023]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:19:30.042411605 (4h52m43.629467826s) [upload=5040]: Starting upload Aug 7 21:19:30.042491373 (4h52m43.629547594s) [upload=5040]: Done starting upload Aug 7 21:19:30.042544984 (4h52m43.629601195s) [upload=5040]: Uploading segments... Aug 7 21:19:30.042585870 (4h52m43.629642091s) [upload=5040]: Got next segment. Inline: true Aug 7 21:19:30.042590809 (4h52m43.62964703s) [upload=5040]: Waiting for error group managing segments... Aug 7 21:19:30.042637226 (4h52m43.629693436s) [upload=5040]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:19:30.042642325 (4h52m43.629698546s) [upload=5040]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:19:30.042647144 (4h52m43.629703365s) [upload=5040]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:20:30.020861737 (4h53m43.607917958s) [upload=5040]: Done uploading segments... Aug 7 21:20:30.020878959 (4h53m43.60793518s) [upload=5040]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:30.129088028 (4h53m43.71614424s) [upload=5055]: Starting upload Aug 7 21:20:30.129144182 (4h53m43.716200404s) [upload=5055]: Done starting upload Aug 7 21:20:30.129191110 (4h53m43.716247332s) [upload=5055]: Uploading segments... Aug 7 21:20:30.129220234 (4h53m43.716276455s) [upload=5055]: Got next segment. Inline: true Aug 7 21:20:30.129223841 (4h53m43.716280062s) [upload=5055]: Waiting for error group managing segments... Aug 7 21:20:30.129266070 (4h53m43.716322282s) [upload=5055]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:30.129269998 (4h53m43.716326219s) [upload=5055]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:20:30.129272523 (4h53m43.716328734s) [upload=5055]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:21:30.117153252 (4h54m43.704209473s) [upload=5055]: Done uploading segments... Aug 7 21:21:30.117172058 (4h54m43.704228279s) [upload=5055]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:30.189934289 (4h54m43.776990511s) [upload=5071]: Starting upload Aug 7 21:21:30.189980476 (4h54m43.777036687s) [upload=5071]: Done starting upload Aug 7 21:21:30.190017424 (4h54m43.777073645s) [upload=5071]: Uploading segments... Aug 7 21:21:30.190043793 (4h54m43.777100005s) [upload=5071]: Got next segment. Inline: true Aug 7 21:21:30.190047100 (4h54m43.777103321s) [upload=5071]: Waiting for error group managing segments... Aug 7 21:21:30.190088458 (4h54m43.777144669s) [upload=5071]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:30.190092425 (4h54m43.777148646s) [upload=5071]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:30.190094709 (4h54m43.77715093s) [upload=5071]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:30.179283539 (4h55m43.76633979s) [upload=5071]: Done uploading segments... Aug 7 21:22:30.179305229 (4h55m43.766361471s) [upload=5071]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:22:44.041348848 (4h55m57.628405089s) [upload=5090]: Starting upload Aug 7 21:22:44.041428967 (4h55m57.628485208s) [upload=5090]: Done starting upload Aug 7 21:22:44.041486454 (4h55m57.628542695s) [upload=5090]: Uploading segments... Aug 7 21:22:44.041542930 (4h55m57.628599171s) [upload=5090]: Got next segment. Inline: true Aug 7 21:22:44.041549963 (4h55m57.628606204s) [upload=5090]: Waiting for error group managing segments... Aug 7 21:22:44.041596389 (4h55m57.628652631s) [upload=5090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:44.041602170 (4h55m57.628658411s) [upload=5090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:44.041605667 (4h55m57.628661917s) [upload=5090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:44.025768658 (4h56m57.612824879s) [upload=5090]: Done uploading segments... Aug 7 21:23:44.025784877 (4h56m57.612841089s) [upload=5090]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.000429744s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...................................................XXXXXXXXX] duration: 30.000657896s FAIL (2025-08-07 21:23:40 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.000677423s core uplink us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 44.265287ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 21:23:31 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 44.29276ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 13 out of 60 is failed (threshold=3) [...............................................XXXXXXXXXXXXX] duration: 30.000730492s FAIL (2025-08-07 21:23:14 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== exceeded Aug 7 21:14:54.825501522 (4h48m8.412557734s) [upload=4964]: Starting upload Aug 7 21:14:54.825580730 (4h48m8.412636951s) [upload=4964]: Done starting upload Aug 7 21:14:54.825629411 (4h48m8.412685632s) [upload=4964]: Uploading segments... Aug 7 21:14:54.825660400 (4h48m8.41271661s) [upload=4964]: Got next segment. Inline: true Aug 7 21:14:54.825664547 (4h48m8.412720758s) [upload=4964]: Waiting for error group managing segments... Aug 7 21:14:54.825705873 (4h48m8.412762115s) [upload=4964]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:54.825711855 (4h48m8.412768076s) [upload=4964]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:14:54.825715662 (4h48m8.412771883s) [upload=4964]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:15:24.812961328 (4h48m38.400017569s) [upload=4964]: Done uploading segments... Aug 7 21:15:24.812988770 (4h48m38.40004501s) [upload=4964]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:15:40.993035983 (4h48m54.580092194s) [upload=4978]: Starting upload Aug 7 21:15:40.993105403 (4h48m54.580161614s) [upload=4978]: Done starting upload Aug 7 21:15:40.993154163 (4h48m54.580210375s) [upload=4978]: Uploading segments... Aug 7 21:15:40.993189519 (4h48m54.580245741s) [upload=4978]: Got next segment. Inline: true Aug 7 21:15:40.993194319 (4h48m54.58025054s) [upload=4978]: Waiting for error group managing segments... Aug 7 21:15:40.993226930 (4h48m54.580283151s) [upload=4978]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:15:40.993231167 (4h48m54.580287379s) [upload=4978]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:40.993234264 (4h48m54.580290474s) [upload=4978]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:16:10.982081529 (4h49m24.569137771s) [upload=4978]: Done uploading segments... Aug 7 21:16:10.982104473 (4h49m24.569160713s) [upload=4978]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:20.227399180 (4h49m33.814455401s) [upload=4990]: Starting upload Aug 7 21:16:20.227450334 (4h49m33.814506545s) [upload=4990]: Done starting upload Aug 7 21:16:20.227491060 (4h49m33.814547272s) [upload=4990]: Uploading segments... Aug 7 21:16:20.227516609 (4h49m33.81457283s) [upload=4990]: Got next segment. Inline: true Aug 7 21:16:20.227520536 (4h49m33.814576757s) [upload=4990]: Waiting for error group managing segments... Aug 7 21:16:20.227552426 (4h49m33.814608636s) [upload=4990]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:16:20.227556263 (4h49m33.814612484s) [upload=4990]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:20.227558527 (4h49m33.814614739s) [upload=4990]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:16:50.216081431 (4h50m3.803137652s) [upload=4990]: Done uploading segments... Aug 7 21:16:50.216097492 (4h50m3.803153713s) [upload=4990]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:57.322533914 (4h50m10.909590156s) [upload=4995]: Starting upload Aug 7 21:16:57.322609355 (4h50m10.909665607s) [upload=4995]: Done starting upload Aug 7 21:16:57.322672883 (4h50m10.909729125s) [upload=4995]: Uploading segments... Aug 7 21:16:57.322711135 (4h50m10.909767376s) [upload=4995]: Got next segment. Inline: true Aug 7 21:16:57.322716685 (4h50m10.909772927s) [upload=4995]: Waiting for error group managing segments... Aug 7 21:16:57.322761279 (4h50m10.90981752s) [upload=4995]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:16:57.322767100 (4h50m10.909823341s) [upload=4995]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:57.322770697 (4h50m10.909826938s) [upload=4995]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:17:27.310589172 (4h50m40.897645383s) [upload=4995]: Done uploading segments... Aug 7 21:17:27.310603289 (4h50m40.897659499s) [upload=4995]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:38.458031408 (4h50m52.045087639s) [upload=5010]: Starting upload Aug 7 21:17:38.458135221 (4h50m52.045191452s) [upload=5010]: Done starting upload Aug 7 21:17:38.458199552 (4h50m52.045255784s) [upload=5010]: Uploading segments... Aug 7 21:17:38.458275263 (4h50m52.045331504s) [upload=5010]: Got next segment. Inline: true Aug 7 21:17:38.458283519 (4h50m52.04533976s) [upload=5010]: Waiting for error group managing segments... Aug 7 21:17:38.458329695 (4h50m52.045385926s) [upload=5010]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:17:38.458335165 (4h50m52.045391407s) [upload=5010]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:38.458338662 (4h50m52.045394893s) [upload=5010]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:18:08.444937554 (4h51m22.031993765s) [upload=5010]: Done uploading segments... Aug 7 21:18:08.444950428 (4h51m22.032006639s) [upload=5010]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:50.205377240 (4h52m3.792433451s) [upload=5028]: Starting upload Aug 7 21:18:50.205432072 (4h52m3.792488293s) [upload=5028]: Done starting upload Aug 7 21:18:50.205472938 (4h52m3.792529149s) [upload=5028]: Uploading segments... Aug 7 21:18:50.205500099 (4h52m3.792556311s) [upload=5028]: Got next segment. Inline: true Aug 7 21:18:50.205503937 (4h52m3.792560147s) [upload=5028]: Waiting for error group managing segments... Aug 7 21:18:50.205531809 (4h52m3.79258802s) [upload=5028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:18:50.205535356 (4h52m3.792591577s) [upload=5028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:50.205538692 (4h52m3.792594913s) [upload=5028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:19:20.193730995 (4h52m33.780787246s) [upload=5028]: Done uploading segments... Aug 7 21:19:20.193750121 (4h52m33.780806372s) [upload=5028]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:03.998495850 (4h53m17.58555209s) [upload=5048]: Starting upload Aug 7 21:20:03.998559819 (4h53m17.585616059s) [upload=5048]: Done starting upload Aug 7 21:20:03.998621224 (4h53m17.585677465s) [upload=5048]: Uploading segments... Aug 7 21:20:03.998661639 (4h53m17.585717879s) [upload=5048]: Got next segment. Inline: true Aug 7 21:20:03.998667419 (4h53m17.58572365s) [upload=5048]: Waiting for error group managing segments... Aug 7 21:20:03.998710790 (4h53m17.585767032s) [upload=5048]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:03.998716000 (4h53m17.585772242s) [upload=5048]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:20:03.998719587 (4h53m17.585775818s) [upload=5048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:20:33.984947930 (4h53m47.572004141s) [upload=5048]: Done uploading segments... Aug 7 21:20:33.985025244 (4h53m47.572081455s) [upload=5048]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:03.692382456 (4h54m17.279438677s) [upload=5064]: Starting upload Aug 7 21:21:03.692428231 (4h54m17.279484453s) [upload=5064]: Done starting upload Aug 7 21:21:03.692467736 (4h54m17.279523957s) [upload=5064]: Uploading segments... Aug 7 21:21:03.692495146 (4h54m17.279551367s) [upload=5064]: Got next segment. Inline: true Aug 7 21:21:03.692500066 (4h54m17.279556287s) [upload=5064]: Waiting for error group managing segments... Aug 7 21:21:03.692529661 (4h54m17.279585882s) [upload=5064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:03.692533137 (4h54m17.279589349s) [upload=5064]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:03.692535481 (4h54m17.279591702s) [upload=5064]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:21:33.679184065 (4h54m47.266240287s) [upload=5064]: Done uploading segments... Aug 7 21:21:33.679200356 (4h54m47.266256568s) [upload=5064]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:41.028727083 (4h54m54.615783293s) [upload=5077]: Starting upload Aug 7 21:21:41.028776044 (4h54m54.615832265s) [upload=5077]: Done starting upload Aug 7 21:21:41.028822871 (4h54m54.615879093s) [upload=5077]: Uploading segments... Aug 7 21:21:41.028849481 (4h54m54.615905693s) [upload=5077]: Got next segment. Inline: true Aug 7 21:21:41.028853158 (4h54m54.61590938s) [upload=5077]: Waiting for error group managing segments... Aug 7 21:21:41.028887172 (4h54m54.615943383s) [upload=5077]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:41.028891160 (4h54m54.615947381s) [upload=5077]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:41.028893564 (4h54m54.615949785s) [upload=5077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:11.020606617 (4h55m24.607662858s) [upload=5077]: Done uploading segments... Aug 7 21:22:11.020629420 (4h55m24.60768566s) [upload=5077]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:22:44.704346518 (4h55m58.291402739s) [upload=5092]: Starting upload Aug 7 21:22:44.704395569 (4h55m58.29145179s) [upload=5092]: Done starting upload Aug 7 21:22:44.704432468 (4h55m58.291488689s) [upload=5092]: Uploading segments... Aug 7 21:22:44.704457685 (4h55m58.291513906s) [upload=5092]: Got next segment. Inline: true Aug 7 21:22:44.704461182 (4h55m58.291517403s) [upload=5092]: Waiting for error group managing segments... Aug 7 21:22:44.704487161 (4h55m58.291543382s) [upload=5092]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:44.704490647 (4h55m58.291546868s) [upload=5092]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:44.704493612 (4h55m58.291549833s) [upload=5092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:14.694775680 (4h56m28.281831901s) [upload=5092]: Done uploading segments... Aug 7 21:23:14.694790848 (4h56m28.281847069s) [upload=5092]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.000741712s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..................................................XXXXXXXXXX] duration: 30.000815198s FAIL (2025-08-07 21:23:11 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 25]: Uploading piece... Aug 7 21:18:54.121608686 (4h52m7.708664927s) [noise=true , address=77.90.31.11:4555 , piece_id=D7YHBQ32VTIJEC4HKWQYHWMOULL3KNN7P7TJSA7LV4TLUICYGWBQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.121894870 (4h52m7.708951121s) [noise=true , address=45.91.116.8:4048 , piece_id=2IZI3WOTOBBKYHKVPJDQ6PGS5VTDAFIGTI373VWKT7QLAZR7W3FA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.121987422 (4h52m7.709043694s) [noise=true , address=77.90.31.19:4841 , piece_id=AUK7XFVC5R73KNHLVVJVRGUTRWBEJ6LP4FZIUJ77DECJRXMHWRRA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.122419388 (4h52m7.70947563s) [noise=true , address=104.36.82.15:4258 , piece_id=VT3VKJFUHTSDCNR7PIBIJXEDYG3HGIVT4JZRR5VM3WJ6A5RHQQZQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.122822722 (4h52m7.709878973s) [noise=true , address=77.90.31.13:4628 , piece_id=N3UNNLVQJ3GALE77VUERGE4PJDIYJK4T5VJEARML2QSMNJ2AAPJA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.123209864 (4h52m7.710266105s) [noise=true , address=45.91.116.5:4297 , piece_id=YK7QNB23S43MPEVNDLAT5WBFAMVBPQ3CVYKEMW5VBUHUAI5W4S5Q, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.123571199 (4h52m7.710627439s) [noise=true , address=104.36.82.14:4223 , piece_id=LX33NZFO2TFV43ARCAIGFYXJZZRR633JWZTHSO6XWSRP7TQXU5HQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.259677990 (4h52m7.846734232s) [upload=5025]: Next returned nil segment Aug 7 21:18:54.259835595 (4h52m7.846891847s) [upload=5025]: Waiting for error group managing segments... Aug 7 21:18:54.269953672 (4h52m7.857009913s) [noise=true , address=45.91.116.7:4011 , piece_id=H2JRFAYC7KLC3TIXQ4PJZTKZLY552JJMZF6XTCFMHFD3RGIUMOCQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.270112598 (4h52m7.857168849s) [noise=true , address=45.91.116.8:4048 , piece_id=2IZI3WOTOBBKYHKVPJDQ6PGS5VTDAFIGTI373VWKT7QLAZR7W3FA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.270947997 (4h52m7.858004239s) [noise=true , address=45.91.116.5:4297 , piece_id=YK7QNB23S43MPEVNDLAT5WBFAMVBPQ3CVYKEMW5VBUHUAI5W4S5Q, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.271046852 (4h52m7.858103094s) [noise=true , address=77.90.31.13:4628 , piece_id=N3UNNLVQJ3GALE77VUERGE4PJDIYJK4T5VJEARML2QSMNJ2AAPJA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.272223499 (4h52m7.85927975s) [noise=true , address=45.91.116.10:4083 , piece_id=HW5YZYWXC7XHQA37JIKXDA7SSO6HOVCCGDS2SHWP7ROPG6MHQ7ZA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.272541713 (4h52m7.859597954s) [noise=true , address=45.91.116.14:4189 , piece_id=GRDSWIHHZ4FEWBEHRY7WPEF546PJWGBIXXCYDESVV4N3GSAWVGYA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.273006260 (4h52m7.860062501s) [noise=true , address=45.91.116.8:4047 , piece_id=7F4MB4X3CJZH3DUXC7VXN3HGQBBRFUV62G4KLCKV7BSARABJENWQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.275575946 (4h52m7.862632188s) [noise=true , address=77.90.31.10:4520 , piece_id=Z5P2H3YFJDC5WMKMFIFVYGFVAU2R27KTNMIAYBDYZGRUT5TWN5QQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.281367938 (4h52m7.868424169s) [noise=true , address=77.90.31.15:4697 , piece_id=PILNZYYWJABT57ITSG77YXQZN7EJKSDHB75RJHE6SPB67FSWCPGQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.284511737 (4h52m7.871567998s) [noise=true , address=77.90.31.19:4841 , piece_id=AUK7XFVC5R73KNHLVVJVRGUTRWBEJ6LP4FZIUJ77DECJRXMHWRRA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.291770877 (4h52m7.878827127s) [noise=true , address=104.36.82.4:4803 , piece_id=DCMHD25G5K5S33FQ72K7VEQ3U6ZV5UBZK5UFVDWUI5I3BX6E4NHA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.298214294 (4h52m7.885270545s) [noise=true , address=77.90.31.11:4555 , piece_id=D7YHBQ32VTIJEC4HKWQYHWMOULL3KNN7P7TJSA7LV4TLUICYGWBQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.532849875 (4h52m8.119906117s) [noise=true , address=104.36.82.6:4876 , piece_id=6WRCDS47O3FUJLE2BT6QV3OOXKGONXLHZ7TMPXKHH4LP2AJGHV5Q, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.532867749 (4h52m8.11992399s) [seg_pos={0 0}, upload=5025]: Segment reached optimal threshold of 13 pieces. Aug 7 21:18:54.533278306 (4h52m8.120334548s) [noise=true , address=104.36.82.15:4259 , piece_id=ZNJ3QCHTB2LJVM7KI52E3RPGDYEHWWFTEEBBNUIOVPIDOQP7VTKQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Aug 7 21:18:54.533410502 (4h52m8.120466743s) [noise=true , address=104.36.82.15:4258 , piece_id=VT3VKJFUHTSDCNR7PIBIJXEDYG3HGIVT4JZRR5VM3WJ6A5RHQQZQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: expected piece hash; context canceled; EOF Aug 7 21:18:54.533558579 (4h52m8.12061482s) [noise=true , address=104.36.82.14:4223 , piece_id=LX33NZFO2TFV43ARCAIGFYXJZZRR633JWZTHSO6XWSRP7TQXU5HQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: write tcp 5.161.89.252:47372->104.36.82.14:4223: use of closed network connection; write tcp 5.161.89.252:47372->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47372->104.36.82.14:4223: use of closed network connection Aug 7 21:18:54.533575210 (4h52m8.120631451s) [seg_pos={0 0}, upload=5025]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:18:54.533666891 (4h52m8.120723132s) [upload=5025]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:18:54.533672582 (4h52m8.120728823s) [upload=5025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:20:54.197308723 (4h54m7.784364965s) [upload=5025]: Done uploading segments... Aug 7 21:20:54.197331416 (4h54m7.784387657s) [upload=5025]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:54.289617342 (4h54m7.876673583s) [upload=5062]: Starting upload Aug 7 21:20:54.289682233 (4h54m7.876738475s) [upload=5062]: Done starting upload Aug 7 21:20:54.290100865 (4h54m7.877157115s) [upload=5062]: Uploading segments... Aug 7 21:20:54.290145758 (4h54m7.877201999s) [upload=5062]: Got next segment. Inline: false Aug 7 21:20:54.290178920 (4h54m7.877235161s) [upload=5062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:54.290184501 (4h54m7.877240741s) [upload=5062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:21:24.274797304 (4h54m37.861853546s) [upload=5062]: Next segment err: context deadline exceeded Aug 7 21:21:24.274996005 (4h54m37.862052246s) [upload=5062]: Done uploading segments... Aug 7 21:21:24.275019569 (4h54m37.86207581s) [upload=5062]: Upload finished. err: context deadline exceeded Aug 7 21:21:31.936040213 (4h54m45.523096464s) [upload=5073]: Starting upload Aug 7 21:21:31.936133387 (4h54m45.523189638s) [upload=5073]: Done starting upload Aug 7 21:21:31.936256627 (4h54m45.523312868s) [upload=5073]: Uploading segments... Aug 7 21:21:31.936495583 (4h54m45.523551824s) [upload=5073]: Got next segment. Inline: false Aug 7 21:21:31.936532813 (4h54m45.523589053s) [upload=5073]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:31.936537992 (4h54m45.523594234s) [upload=5073]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:22:01.921915749 (4h55m15.50897196s) [upload=5073]: Next segment err: context deadline exceeded Aug 7 21:22:01.922047263 (4h55m15.509103475s) [upload=5073]: Done uploading segments... Aug 7 21:22:01.922079755 (4h55m15.509135976s) [upload=5073]: Upload finished. err: context deadline exceeded Aug 7 21:22:02.000538540 (4h55m15.587594761s) [upload=5082]: Starting upload Aug 7 21:22:02.000624420 (4h55m15.587680652s) [upload=5082]: Done starting upload Aug 7 21:22:02.000735858 (4h55m15.587792089s) [upload=5082]: Uploading segments... Aug 7 21:22:02.000881511 (4h55m15.587937761s) [upload=5082]: Got next segment. Inline: false Aug 7 21:22:02.000947453 (4h55m15.588003684s) [upload=5082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:02.000954897 (4h55m15.588011118s) [upload=5082]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:22:31.979343661 (4h55m45.566399882s) [upload=5082]: Next segment err: context deadline exceeded Aug 7 21:22:31.979438208 (4h55m45.566494418s) [upload=5082]: Done uploading segments... Aug 7 21:22:31.979474655 (4h55m45.566530877s) [upload=5082]: Upload finished. err: context deadline exceeded Aug 7 21:22:41.968187982 (4h55m55.555244194s) [upload=5089]: Starting upload Aug 7 21:22:41.968245641 (4h55m55.555301862s) [upload=5089]: Done starting upload Aug 7 21:22:41.968339846 (4h55m55.555396087s) [upload=5089]: Uploading segments... Aug 7 21:22:41.968596806 (4h55m55.555653047s) [upload=5089]: Got next segment. Inline: false Aug 7 21:22:41.968638122 (4h55m55.555694354s) [upload=5089]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:41.968644665 (4h55m55.555700906s) [upload=5089]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:23:11.943981705 (4h56m25.531037956s) [upload=5089]: Next segment err: context deadline exceeded Aug 7 21:23:11.944159476 (4h56m25.531215718s) [upload=5089]: Done uploading segments... Aug 7 21:23:11.944182590 (4h56m25.53123883s) [upload=5089]: Upload finished. err: context deadline exceeded Duration: 30.000835848s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 783.409477ms FAIL (2025-08-07 21:23:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 783.430266ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [................................................XXXXXXXXXXXX] duration: 30.004203744s FAIL (2025-08-07 21:23:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.004224341s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 10 out of 60 is failed (threshold=3) [.................................................XX.XXXXXXXX] duration: 1m0.000410978s FAIL (2025-08-07 21:22:58 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000429553s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 14 out of 60 is failed (threshold=3) [............................................XX..XXXXXXXXXXXX] duration: 30.000551758s FAIL (2025-08-07 21:22:47 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.000576304s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 11 out of 60 is failed (threshold=3) [.................................................XXXXXXXXXXX] duration: 30.000506406s FAIL (2025-08-07 21:22:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000517908s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 710.544563ms FAIL (2025-08-07 21:22:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 710.570192ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...................................................XXXXXXXXX] duration: 30.001415446s FAIL (2025-08-07 21:22:31 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== I52E3RPGDYEHWWFTEEBBNUIOVPIDOQP7VTKQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.120776011 (4h52m7.707832252s) [noise=true , address=77.90.31.15:4697 , piece_id=PILNZYYWJABT57ITSG77YXQZN7EJKSDHB75RJHE6SPB67FSWCPGQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.120880315 (4h52m7.707936557s) [noise=true , address=45.91.116.14:4189 , piece_id=GRDSWIHHZ4FEWBEHRY7WPEF546PJWGBIXXCYDESVV4N3GSAWVGYA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.121253463 (4h52m7.708309713s) [noise=true , address=45.91.116.10:4083 , piece_id=HW5YZYWXC7XHQA37JIKXDA7SSO6HOVCCGDS2SHWP7ROPG6MHQ7ZA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.121362616 (4h52m7.708418858s) [noise=true , address=77.90.31.10:4520 , piece_id=Z5P2H3YFJDC5WMKMFIFVYGFVAU2R27KTNMIAYBDYZGRUT5TWN5QQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.121608686 (4h52m7.708664927s) [noise=true , address=77.90.31.11:4555 , piece_id=D7YHBQ32VTIJEC4HKWQYHWMOULL3KNN7P7TJSA7LV4TLUICYGWBQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.121894870 (4h52m7.708951121s) [noise=true , address=45.91.116.8:4048 , piece_id=2IZI3WOTOBBKYHKVPJDQ6PGS5VTDAFIGTI373VWKT7QLAZR7W3FA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.121987422 (4h52m7.709043694s) [noise=true , address=77.90.31.19:4841 , piece_id=AUK7XFVC5R73KNHLVVJVRGUTRWBEJ6LP4FZIUJ77DECJRXMHWRRA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.122419388 (4h52m7.70947563s) [noise=true , address=104.36.82.15:4258 , piece_id=VT3VKJFUHTSDCNR7PIBIJXEDYG3HGIVT4JZRR5VM3WJ6A5RHQQZQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.122822722 (4h52m7.709878973s) [noise=true , address=77.90.31.13:4628 , piece_id=N3UNNLVQJ3GALE77VUERGE4PJDIYJK4T5VJEARML2QSMNJ2AAPJA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.123209864 (4h52m7.710266105s) [noise=true , address=45.91.116.5:4297 , piece_id=YK7QNB23S43MPEVNDLAT5WBFAMVBPQ3CVYKEMW5VBUHUAI5W4S5Q, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.123571199 (4h52m7.710627439s) [noise=true , address=104.36.82.14:4223 , piece_id=LX33NZFO2TFV43ARCAIGFYXJZZRR633JWZTHSO6XWSRP7TQXU5HQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.259677990 (4h52m7.846734232s) [upload=5025]: Next returned nil segment Aug 7 21:18:54.259835595 (4h52m7.846891847s) [upload=5025]: Waiting for error group managing segments... Aug 7 21:18:54.269953672 (4h52m7.857009913s) [noise=true , address=45.91.116.7:4011 , piece_id=H2JRFAYC7KLC3TIXQ4PJZTKZLY552JJMZF6XTCFMHFD3RGIUMOCQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.270112598 (4h52m7.857168849s) [noise=true , address=45.91.116.8:4048 , piece_id=2IZI3WOTOBBKYHKVPJDQ6PGS5VTDAFIGTI373VWKT7QLAZR7W3FA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.270947997 (4h52m7.858004239s) [noise=true , address=45.91.116.5:4297 , piece_id=YK7QNB23S43MPEVNDLAT5WBFAMVBPQ3CVYKEMW5VBUHUAI5W4S5Q, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.271046852 (4h52m7.858103094s) [noise=true , address=77.90.31.13:4628 , piece_id=N3UNNLVQJ3GALE77VUERGE4PJDIYJK4T5VJEARML2QSMNJ2AAPJA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.272223499 (4h52m7.85927975s) [noise=true , address=45.91.116.10:4083 , piece_id=HW5YZYWXC7XHQA37JIKXDA7SSO6HOVCCGDS2SHWP7ROPG6MHQ7ZA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.272541713 (4h52m7.859597954s) [noise=true , address=45.91.116.14:4189 , piece_id=GRDSWIHHZ4FEWBEHRY7WPEF546PJWGBIXXCYDESVV4N3GSAWVGYA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.273006260 (4h52m7.860062501s) [noise=true , address=45.91.116.8:4047 , piece_id=7F4MB4X3CJZH3DUXC7VXN3HGQBBRFUV62G4KLCKV7BSARABJENWQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.275575946 (4h52m7.862632188s) [noise=true , address=77.90.31.10:4520 , piece_id=Z5P2H3YFJDC5WMKMFIFVYGFVAU2R27KTNMIAYBDYZGRUT5TWN5QQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.281367938 (4h52m7.868424169s) [noise=true , address=77.90.31.15:4697 , piece_id=PILNZYYWJABT57ITSG77YXQZN7EJKSDHB75RJHE6SPB67FSWCPGQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.284511737 (4h52m7.871567998s) [noise=true , address=77.90.31.19:4841 , piece_id=AUK7XFVC5R73KNHLVVJVRGUTRWBEJ6LP4FZIUJ77DECJRXMHWRRA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.291770877 (4h52m7.878827127s) [noise=true , address=104.36.82.4:4803 , piece_id=DCMHD25G5K5S33FQ72K7VEQ3U6ZV5UBZK5UFVDWUI5I3BX6E4NHA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.298214294 (4h52m7.885270545s) [noise=true , address=77.90.31.11:4555 , piece_id=D7YHBQ32VTIJEC4HKWQYHWMOULL3KNN7P7TJSA7LV4TLUICYGWBQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.532849875 (4h52m8.119906117s) [noise=true , address=104.36.82.6:4876 , piece_id=6WRCDS47O3FUJLE2BT6QV3OOXKGONXLHZ7TMPXKHH4LP2AJGHV5Q, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.532867749 (4h52m8.11992399s) [seg_pos={0 0}, upload=5025]: Segment reached optimal threshold of 13 pieces. Aug 7 21:18:54.533278306 (4h52m8.120334548s) [noise=true , address=104.36.82.15:4259 , piece_id=ZNJ3QCHTB2LJVM7KI52E3RPGDYEHWWFTEEBBNUIOVPIDOQP7VTKQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Aug 7 21:18:54.533410502 (4h52m8.120466743s) [noise=true , address=104.36.82.15:4258 , piece_id=VT3VKJFUHTSDCNR7PIBIJXEDYG3HGIVT4JZRR5VM3WJ6A5RHQQZQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: expected piece hash; context canceled; EOF Aug 7 21:18:54.533558579 (4h52m8.12061482s) [noise=true , address=104.36.82.14:4223 , piece_id=LX33NZFO2TFV43ARCAIGFYXJZZRR633JWZTHSO6XWSRP7TQXU5HQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: write tcp 5.161.89.252:47372->104.36.82.14:4223: use of closed network connection; write tcp 5.161.89.252:47372->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47372->104.36.82.14:4223: use of closed network connection Aug 7 21:18:54.533575210 (4h52m8.120631451s) [seg_pos={0 0}, upload=5025]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:18:54.533666891 (4h52m8.120723132s) [upload=5025]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:18:54.533672582 (4h52m8.120728823s) [upload=5025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:20:54.197308723 (4h54m7.784364965s) [upload=5025]: Done uploading segments... Aug 7 21:20:54.197331416 (4h54m7.784387657s) [upload=5025]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:54.289617342 (4h54m7.876673583s) [upload=5062]: Starting upload Aug 7 21:20:54.289682233 (4h54m7.876738475s) [upload=5062]: Done starting upload Aug 7 21:20:54.290100865 (4h54m7.877157115s) [upload=5062]: Uploading segments... Aug 7 21:20:54.290145758 (4h54m7.877201999s) [upload=5062]: Got next segment. Inline: false Aug 7 21:20:54.290178920 (4h54m7.877235161s) [upload=5062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:54.290184501 (4h54m7.877240741s) [upload=5062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:21:24.274797304 (4h54m37.861853546s) [upload=5062]: Next segment err: context deadline exceeded Aug 7 21:21:24.274996005 (4h54m37.862052246s) [upload=5062]: Done uploading segments... Aug 7 21:21:24.275019569 (4h54m37.86207581s) [upload=5062]: Upload finished. err: context deadline exceeded Aug 7 21:21:31.936040213 (4h54m45.523096464s) [upload=5073]: Starting upload Aug 7 21:21:31.936133387 (4h54m45.523189638s) [upload=5073]: Done starting upload Aug 7 21:21:31.936256627 (4h54m45.523312868s) [upload=5073]: Uploading segments... Aug 7 21:21:31.936495583 (4h54m45.523551824s) [upload=5073]: Got next segment. Inline: false Aug 7 21:21:31.936532813 (4h54m45.523589053s) [upload=5073]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:31.936537992 (4h54m45.523594234s) [upload=5073]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:22:01.921915749 (4h55m15.50897196s) [upload=5073]: Next segment err: context deadline exceeded Aug 7 21:22:01.922047263 (4h55m15.509103475s) [upload=5073]: Done uploading segments... Aug 7 21:22:01.922079755 (4h55m15.509135976s) [upload=5073]: Upload finished. err: context deadline exceeded Aug 7 21:22:02.000538540 (4h55m15.587594761s) [upload=5082]: Starting upload Aug 7 21:22:02.000624420 (4h55m15.587680652s) [upload=5082]: Done starting upload Aug 7 21:22:02.000735858 (4h55m15.587792089s) [upload=5082]: Uploading segments... Aug 7 21:22:02.000881511 (4h55m15.587937761s) [upload=5082]: Got next segment. Inline: false Aug 7 21:22:02.000947453 (4h55m15.588003684s) [upload=5082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:02.000954897 (4h55m15.588011118s) [upload=5082]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:22:31.979343661 (4h55m45.566399882s) [upload=5082]: Next segment err: context deadline exceeded Aug 7 21:22:31.979438208 (4h55m45.566494418s) [upload=5082]: Done uploading segments... Aug 7 21:22:31.979474655 (4h55m45.566530877s) [upload=5082]: Upload finished. err: context deadline exceeded Duration: 30.001427108s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 8 out of 60 is failed (threshold=3) [....................................................XXXXXXXX] duration: 30.000397417s FAIL (2025-08-07 21:22:30 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.000416733s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 8 out of 60 is failed (threshold=3) [....................................................XXXXXXXX] duration: 1m0.001090526s FAIL (2025-08-07 21:22:30 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 094 (4h45m5.417729315s) [upload=4915]: Upload finished. err: <nil> Aug 7 21:12:54.778494646 (4h46m8.365550856s) [upload=4932]: Starting upload Aug 7 21:12:54.778560719 (4h46m8.36561693s) [upload=4932]: Done starting upload Aug 7 21:12:54.778608638 (4h46m8.365664859s) [upload=4932]: Uploading segments... Aug 7 21:12:54.778637322 (4h46m8.365693543s) [upload=4932]: Got next segment. Inline: true Aug 7 21:12:54.778640999 (4h46m8.36569722s) [upload=4932]: Waiting for error group managing segments... Aug 7 21:12:54.778677527 (4h46m8.365733739s) [upload=4932]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:12:54.778681655 (4h46m8.365737876s) [upload=4932]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:12:54.778684771 (4h46m8.365740983s) [upload=4932]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:12:54.868239378 (4h46m8.455295599s) [upload=4932]: Done uploading segments... Aug 7 21:12:54.868253865 (4h46m8.455310086s) [upload=4932]: Upload finished. err: <nil> Aug 7 21:13:23.238906839 (4h46m36.82596306s) [upload=4942]: Starting upload Aug 7 21:13:23.238959356 (4h46m36.826015567s) [upload=4942]: Done starting upload Aug 7 21:13:23.239011013 (4h46m36.826067234s) [upload=4942]: Uploading segments... Aug 7 21:13:23.239039727 (4h46m36.826095937s) [upload=4942]: Got next segment. Inline: true Aug 7 21:13:23.239043483 (4h46m36.826099695s) [upload=4942]: Waiting for error group managing segments... Aug 7 21:13:23.239098526 (4h46m36.826154737s) [upload=4942]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:13:23.239104707 (4h46m36.826160919s) [upload=4942]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:13:23.239108474 (4h46m36.826164696s) [upload=4942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:13:23.309357825 (4h46m36.896414127s) [upload=4942]: Done uploading segments... Aug 7 21:13:23.309383974 (4h46m36.896440216s) [upload=4942]: Upload finished. err: <nil> Aug 7 21:14:18.294119361 (4h47m31.881175572s) [upload=4957]: Starting upload Aug 7 21:14:18.294187468 (4h47m31.881243679s) [upload=4957]: Done starting upload Aug 7 21:14:18.294261516 (4h47m31.881317737s) [upload=4957]: Uploading segments... Aug 7 21:14:18.294305068 (4h47m31.881361289s) [upload=4957]: Got next segment. Inline: true Aug 7 21:14:18.294311409 (4h47m31.881367621s) [upload=4957]: Waiting for error group managing segments... Aug 7 21:14:18.294349491 (4h47m31.881405712s) [upload=4957]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:18.294354059 (4h47m31.88141028s) [upload=4957]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:14:18.294356643 (4h47m31.881412854s) [upload=4957]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:15:18.280461255 (4h48m31.867517496s) [upload=4957]: Done uploading segments... Aug 7 21:15:18.280484308 (4h48m31.867540559s) [upload=4957]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:15:18.387536247 (4h48m31.974592488s) [upload=4974]: Starting upload Aug 7 21:15:18.387607770 (4h48m31.974664011s) [upload=4974]: Done starting upload Aug 7 21:15:18.387666109 (4h48m31.974722361s) [upload=4974]: Uploading segments... Aug 7 21:15:18.387706996 (4h48m31.974763236s) [upload=4974]: Got next segment. Inline: true Aug 7 21:15:18.387712766 (4h48m31.974769007s) [upload=4974]: Waiting for error group managing segments... Aug 7 21:15:18.387755796 (4h48m31.974812038s) [upload=4974]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:15:18.387761227 (4h48m31.974817468s) [upload=4974]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:18.387764853 (4h48m31.974821095s) [upload=4974]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:16:18.373970870 (4h49m31.961027121s) [upload=4974]: Done uploading segments... Aug 7 21:16:18.374035110 (4h49m31.961091352s) [upload=4974]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:18.454541239 (4h49m32.04159748s) [upload=4989]: Starting upload Aug 7 21:16:18.454621288 (4h49m32.041677529s) [upload=4989]: Done starting upload Aug 7 21:16:18.454699754 (4h49m32.041756005s) [upload=4989]: Uploading segments... Aug 7 21:16:18.454758153 (4h49m32.041814395s) [upload=4989]: Got next segment. Inline: true Aug 7 21:16:18.454765147 (4h49m32.041821398s) [upload=4989]: Waiting for error group managing segments... Aug 7 21:16:18.454813777 (4h49m32.041870018s) [upload=4989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:16:18.454819818 (4h49m32.04187606s) [upload=4989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:18.454823425 (4h49m32.041879666s) [upload=4989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:17:18.435319201 (4h50m32.022375413s) [upload=4989]: Done uploading segments... Aug 7 21:17:18.435335041 (4h50m32.022391262s) [upload=4989]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:29.913551414 (4h50m43.500607635s) [upload=5005]: Starting upload Aug 7 21:17:29.913606246 (4h50m43.500662467s) [upload=5005]: Done starting upload Aug 7 21:17:29.913653174 (4h50m43.500709395s) [upload=5005]: Uploading segments... Aug 7 21:17:29.913680986 (4h50m43.500737207s) [upload=5005]: Got next segment. Inline: true Aug 7 21:17:29.913684613 (4h50m43.500740835s) [upload=5005]: Waiting for error group managing segments... Aug 7 21:17:29.913715249 (4h50m43.500771471s) [upload=5005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:17:29.913719057 (4h50m43.500775278s) [upload=5005]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:29.913721331 (4h50m43.500777542s) [upload=5005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:18:29.900370534 (4h51m43.487426745s) [upload=5005]: Done uploading segments... Aug 7 21:18:29.900390291 (4h51m43.487446542s) [upload=5005]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:29.988006379 (4h51m43.5750626s) [upload=5023]: Starting upload Aug 7 21:18:29.988053607 (4h51m43.575109818s) [upload=5023]: Done starting upload Aug 7 21:18:29.988109782 (4h51m43.575166003s) [upload=5023]: Uploading segments... Aug 7 21:18:29.988142012 (4h51m43.575198224s) [upload=5023]: Got next segment. Inline: true Aug 7 21:18:29.988147753 (4h51m43.575203973s) [upload=5023]: Waiting for error group managing segments... Aug 7 21:18:29.988197475 (4h51m43.575253687s) [upload=5023]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:18:29.988203116 (4h51m43.575259327s) [upload=5023]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:29.988206943 (4h51m43.575263164s) [upload=5023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:19:29.967158594 (4h52m43.554214836s) [upload=5023]: Done uploading segments... Aug 7 21:19:29.967184642 (4h52m43.554240884s) [upload=5023]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:19:30.042411605 (4h52m43.629467826s) [upload=5040]: Starting upload Aug 7 21:19:30.042491373 (4h52m43.629547594s) [upload=5040]: Done starting upload Aug 7 21:19:30.042544984 (4h52m43.629601195s) [upload=5040]: Uploading segments... Aug 7 21:19:30.042585870 (4h52m43.629642091s) [upload=5040]: Got next segment. Inline: true Aug 7 21:19:30.042590809 (4h52m43.62964703s) [upload=5040]: Waiting for error group managing segments... Aug 7 21:19:30.042637226 (4h52m43.629693436s) [upload=5040]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:19:30.042642325 (4h52m43.629698546s) [upload=5040]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:19:30.042647144 (4h52m43.629703365s) [upload=5040]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:20:30.020861737 (4h53m43.607917958s) [upload=5040]: Done uploading segments... Aug 7 21:20:30.020878959 (4h53m43.60793518s) [upload=5040]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:30.129088028 (4h53m43.71614424s) [upload=5055]: Starting upload Aug 7 21:20:30.129144182 (4h53m43.716200404s) [upload=5055]: Done starting upload Aug 7 21:20:30.129191110 (4h53m43.716247332s) [upload=5055]: Uploading segments... Aug 7 21:20:30.129220234 (4h53m43.716276455s) [upload=5055]: Got next segment. Inline: true Aug 7 21:20:30.129223841 (4h53m43.716280062s) [upload=5055]: Waiting for error group managing segments... Aug 7 21:20:30.129266070 (4h53m43.716322282s) [upload=5055]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:30.129269998 (4h53m43.716326219s) [upload=5055]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:20:30.129272523 (4h53m43.716328734s) [upload=5055]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:21:30.117153252 (4h54m43.704209473s) [upload=5055]: Done uploading segments... Aug 7 21:21:30.117172058 (4h54m43.704228279s) [upload=5055]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:30.189934289 (4h54m43.776990511s) [upload=5071]: Starting upload Aug 7 21:21:30.189980476 (4h54m43.777036687s) [upload=5071]: Done starting upload Aug 7 21:21:30.190017424 (4h54m43.777073645s) [upload=5071]: Uploading segments... Aug 7 21:21:30.190043793 (4h54m43.777100005s) [upload=5071]: Got next segment. Inline: true Aug 7 21:21:30.190047100 (4h54m43.777103321s) [upload=5071]: Waiting for error group managing segments... Aug 7 21:21:30.190088458 (4h54m43.777144669s) [upload=5071]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:30.190092425 (4h54m43.777148646s) [upload=5071]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:30.190094709 (4h54m43.77715093s) [upload=5071]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:30.179283539 (4h55m43.76633979s) [upload=5071]: Done uploading segments... Aug 7 21:22:30.179305229 (4h55m43.766361471s) [upload=5071]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.00110378s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 25.971107ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 21:22:28 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 25.999352ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 10 out of 60 is failed (threshold=3) [..................................................XXXXXXXXXX] duration: 30.001319235s FAIL (2025-08-07 21:22:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001495112s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 12 out of 60 is failed (threshold=3) [................................................XXXXXXXXXXXX] duration: 30.001627108s FAIL (2025-08-07 21:22:11 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== exceeded Aug 7 21:14:13.522007492 (4h47m27.109063713s) [upload=4955]: Starting upload Aug 7 21:14:13.522102209 (4h47m27.10915843s) [upload=4955]: Done starting upload Aug 7 21:14:13.522154938 (4h47m27.109211179s) [upload=4955]: Uploading segments... Aug 7 21:14:13.522235488 (4h47m27.109291729s) [upload=4955]: Got next segment. Inline: true Aug 7 21:14:13.522242682 (4h47m27.109298922s) [upload=4955]: Waiting for error group managing segments... Aug 7 21:14:13.522293467 (4h47m27.109349708s) [upload=4955]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:13.522299518 (4h47m27.109355759s) [upload=4955]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:14:13.522303275 (4h47m27.109359516s) [upload=4955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:14:43.507883903 (4h47m57.094940144s) [upload=4955]: Done uploading segments... Aug 7 21:14:43.507905914 (4h47m57.094962156s) [upload=4955]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:14:54.825501522 (4h48m8.412557734s) [upload=4964]: Starting upload Aug 7 21:14:54.825580730 (4h48m8.412636951s) [upload=4964]: Done starting upload Aug 7 21:14:54.825629411 (4h48m8.412685632s) [upload=4964]: Uploading segments... Aug 7 21:14:54.825660400 (4h48m8.41271661s) [upload=4964]: Got next segment. Inline: true Aug 7 21:14:54.825664547 (4h48m8.412720758s) [upload=4964]: Waiting for error group managing segments... Aug 7 21:14:54.825705873 (4h48m8.412762115s) [upload=4964]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:54.825711855 (4h48m8.412768076s) [upload=4964]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:14:54.825715662 (4h48m8.412771883s) [upload=4964]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:15:24.812961328 (4h48m38.400017569s) [upload=4964]: Done uploading segments... Aug 7 21:15:24.812988770 (4h48m38.40004501s) [upload=4964]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:15:40.993035983 (4h48m54.580092194s) [upload=4978]: Starting upload Aug 7 21:15:40.993105403 (4h48m54.580161614s) [upload=4978]: Done starting upload Aug 7 21:15:40.993154163 (4h48m54.580210375s) [upload=4978]: Uploading segments... Aug 7 21:15:40.993189519 (4h48m54.580245741s) [upload=4978]: Got next segment. Inline: true Aug 7 21:15:40.993194319 (4h48m54.58025054s) [upload=4978]: Waiting for error group managing segments... Aug 7 21:15:40.993226930 (4h48m54.580283151s) [upload=4978]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:15:40.993231167 (4h48m54.580287379s) [upload=4978]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:40.993234264 (4h48m54.580290474s) [upload=4978]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:16:10.982081529 (4h49m24.569137771s) [upload=4978]: Done uploading segments... Aug 7 21:16:10.982104473 (4h49m24.569160713s) [upload=4978]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:20.227399180 (4h49m33.814455401s) [upload=4990]: Starting upload Aug 7 21:16:20.227450334 (4h49m33.814506545s) [upload=4990]: Done starting upload Aug 7 21:16:20.227491060 (4h49m33.814547272s) [upload=4990]: Uploading segments... Aug 7 21:16:20.227516609 (4h49m33.81457283s) [upload=4990]: Got next segment. Inline: true Aug 7 21:16:20.227520536 (4h49m33.814576757s) [upload=4990]: Waiting for error group managing segments... Aug 7 21:16:20.227552426 (4h49m33.814608636s) [upload=4990]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:16:20.227556263 (4h49m33.814612484s) [upload=4990]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:20.227558527 (4h49m33.814614739s) [upload=4990]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:16:50.216081431 (4h50m3.803137652s) [upload=4990]: Done uploading segments... Aug 7 21:16:50.216097492 (4h50m3.803153713s) [upload=4990]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:57.322533914 (4h50m10.909590156s) [upload=4995]: Starting upload Aug 7 21:16:57.322609355 (4h50m10.909665607s) [upload=4995]: Done starting upload Aug 7 21:16:57.322672883 (4h50m10.909729125s) [upload=4995]: Uploading segments... Aug 7 21:16:57.322711135 (4h50m10.909767376s) [upload=4995]: Got next segment. Inline: true Aug 7 21:16:57.322716685 (4h50m10.909772927s) [upload=4995]: Waiting for error group managing segments... Aug 7 21:16:57.322761279 (4h50m10.90981752s) [upload=4995]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:16:57.322767100 (4h50m10.909823341s) [upload=4995]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:57.322770697 (4h50m10.909826938s) [upload=4995]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:17:27.310589172 (4h50m40.897645383s) [upload=4995]: Done uploading segments... Aug 7 21:17:27.310603289 (4h50m40.897659499s) [upload=4995]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:38.458031408 (4h50m52.045087639s) [upload=5010]: Starting upload Aug 7 21:17:38.458135221 (4h50m52.045191452s) [upload=5010]: Done starting upload Aug 7 21:17:38.458199552 (4h50m52.045255784s) [upload=5010]: Uploading segments... Aug 7 21:17:38.458275263 (4h50m52.045331504s) [upload=5010]: Got next segment. Inline: true Aug 7 21:17:38.458283519 (4h50m52.04533976s) [upload=5010]: Waiting for error group managing segments... Aug 7 21:17:38.458329695 (4h50m52.045385926s) [upload=5010]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:17:38.458335165 (4h50m52.045391407s) [upload=5010]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:38.458338662 (4h50m52.045394893s) [upload=5010]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:18:08.444937554 (4h51m22.031993765s) [upload=5010]: Done uploading segments... Aug 7 21:18:08.444950428 (4h51m22.032006639s) [upload=5010]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:50.205377240 (4h52m3.792433451s) [upload=5028]: Starting upload Aug 7 21:18:50.205432072 (4h52m3.792488293s) [upload=5028]: Done starting upload Aug 7 21:18:50.205472938 (4h52m3.792529149s) [upload=5028]: Uploading segments... Aug 7 21:18:50.205500099 (4h52m3.792556311s) [upload=5028]: Got next segment. Inline: true Aug 7 21:18:50.205503937 (4h52m3.792560147s) [upload=5028]: Waiting for error group managing segments... Aug 7 21:18:50.205531809 (4h52m3.79258802s) [upload=5028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:18:50.205535356 (4h52m3.792591577s) [upload=5028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:50.205538692 (4h52m3.792594913s) [upload=5028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:19:20.193730995 (4h52m33.780787246s) [upload=5028]: Done uploading segments... Aug 7 21:19:20.193750121 (4h52m33.780806372s) [upload=5028]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:03.998495850 (4h53m17.58555209s) [upload=5048]: Starting upload Aug 7 21:20:03.998559819 (4h53m17.585616059s) [upload=5048]: Done starting upload Aug 7 21:20:03.998621224 (4h53m17.585677465s) [upload=5048]: Uploading segments... Aug 7 21:20:03.998661639 (4h53m17.585717879s) [upload=5048]: Got next segment. Inline: true Aug 7 21:20:03.998667419 (4h53m17.58572365s) [upload=5048]: Waiting for error group managing segments... Aug 7 21:20:03.998710790 (4h53m17.585767032s) [upload=5048]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:03.998716000 (4h53m17.585772242s) [upload=5048]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:20:03.998719587 (4h53m17.585775818s) [upload=5048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:20:33.984947930 (4h53m47.572004141s) [upload=5048]: Done uploading segments... Aug 7 21:20:33.985025244 (4h53m47.572081455s) [upload=5048]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:03.692382456 (4h54m17.279438677s) [upload=5064]: Starting upload Aug 7 21:21:03.692428231 (4h54m17.279484453s) [upload=5064]: Done starting upload Aug 7 21:21:03.692467736 (4h54m17.279523957s) [upload=5064]: Uploading segments... Aug 7 21:21:03.692495146 (4h54m17.279551367s) [upload=5064]: Got next segment. Inline: true Aug 7 21:21:03.692500066 (4h54m17.279556287s) [upload=5064]: Waiting for error group managing segments... Aug 7 21:21:03.692529661 (4h54m17.279585882s) [upload=5064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:03.692533137 (4h54m17.279589349s) [upload=5064]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:03.692535481 (4h54m17.279591702s) [upload=5064]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:21:33.679184065 (4h54m47.266240287s) [upload=5064]: Done uploading segments... Aug 7 21:21:33.679200356 (4h54m47.266256568s) [upload=5064]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:41.028727083 (4h54m54.615783293s) [upload=5077]: Starting upload Aug 7 21:21:41.028776044 (4h54m54.615832265s) [upload=5077]: Done starting upload Aug 7 21:21:41.028822871 (4h54m54.615879093s) [upload=5077]: Uploading segments... Aug 7 21:21:41.028849481 (4h54m54.615905693s) [upload=5077]: Got next segment. Inline: true Aug 7 21:21:41.028853158 (4h54m54.61590938s) [upload=5077]: Waiting for error group managing segments... Aug 7 21:21:41.028887172 (4h54m54.615943383s) [upload=5077]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:41.028891160 (4h54m54.615947381s) [upload=5077]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:41.028893564 (4h54m54.615949785s) [upload=5077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:11.020606617 (4h55m24.607662858s) [upload=5077]: Done uploading segments... Aug 7 21:22:11.020629420 (4h55m24.60768566s) [upload=5077]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.001647976s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 13 out of 60 is failed (threshold=3) [.............................................XX..XXXXXXXXXXX] duration: 30.00109542s FAIL (2025-08-07 21:22:04 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001115548s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [....................................................XXXXXXXX] duration: 30.000577619s FAIL (2025-08-07 21:22:01 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== se=true , address=45.91.116.7:4011 , piece_id=H2JRFAYC7KLC3TIXQ4PJZTKZLY552JJMZF6XTCFMHFD3RGIUMOCQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.119574769 (4h52m7.70663101s) [noise=true , address=45.91.116.8:4047 , piece_id=7F4MB4X3CJZH3DUXC7VXN3HGQBBRFUV62G4KLCKV7BSARABJENWQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.120057159 (4h52m7.7071134s) [noise=true , address=104.36.82.4:4803 , piece_id=DCMHD25G5K5S33FQ72K7VEQ3U6ZV5UBZK5UFVDWUI5I3BX6E4NHA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.120241273 (4h52m7.707297534s) [noise=true , address=104.36.82.6:4876 , piece_id=6WRCDS47O3FUJLE2BT6QV3OOXKGONXLHZ7TMPXKHH4LP2AJGHV5Q, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.120508351 (4h52m7.707564592s) [noise=true , address=104.36.82.15:4259 , piece_id=ZNJ3QCHTB2LJVM7KI52E3RPGDYEHWWFTEEBBNUIOVPIDOQP7VTKQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.120776011 (4h52m7.707832252s) [noise=true , address=77.90.31.15:4697 , piece_id=PILNZYYWJABT57ITSG77YXQZN7EJKSDHB75RJHE6SPB67FSWCPGQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.120880315 (4h52m7.707936557s) [noise=true , address=45.91.116.14:4189 , piece_id=GRDSWIHHZ4FEWBEHRY7WPEF546PJWGBIXXCYDESVV4N3GSAWVGYA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.121253463 (4h52m7.708309713s) [noise=true , address=45.91.116.10:4083 , piece_id=HW5YZYWXC7XHQA37JIKXDA7SSO6HOVCCGDS2SHWP7ROPG6MHQ7ZA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.121362616 (4h52m7.708418858s) [noise=true , address=77.90.31.10:4520 , piece_id=Z5P2H3YFJDC5WMKMFIFVYGFVAU2R27KTNMIAYBDYZGRUT5TWN5QQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.121608686 (4h52m7.708664927s) [noise=true , address=77.90.31.11:4555 , piece_id=D7YHBQ32VTIJEC4HKWQYHWMOULL3KNN7P7TJSA7LV4TLUICYGWBQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.121894870 (4h52m7.708951121s) [noise=true , address=45.91.116.8:4048 , piece_id=2IZI3WOTOBBKYHKVPJDQ6PGS5VTDAFIGTI373VWKT7QLAZR7W3FA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.121987422 (4h52m7.709043694s) [noise=true , address=77.90.31.19:4841 , piece_id=AUK7XFVC5R73KNHLVVJVRGUTRWBEJ6LP4FZIUJ77DECJRXMHWRRA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.122419388 (4h52m7.70947563s) [noise=true , address=104.36.82.15:4258 , piece_id=VT3VKJFUHTSDCNR7PIBIJXEDYG3HGIVT4JZRR5VM3WJ6A5RHQQZQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.122822722 (4h52m7.709878973s) [noise=true , address=77.90.31.13:4628 , piece_id=N3UNNLVQJ3GALE77VUERGE4PJDIYJK4T5VJEARML2QSMNJ2AAPJA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.123209864 (4h52m7.710266105s) [noise=true , address=45.91.116.5:4297 , piece_id=YK7QNB23S43MPEVNDLAT5WBFAMVBPQ3CVYKEMW5VBUHUAI5W4S5Q, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.123571199 (4h52m7.710627439s) [noise=true , address=104.36.82.14:4223 , piece_id=LX33NZFO2TFV43ARCAIGFYXJZZRR633JWZTHSO6XWSRP7TQXU5HQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.259677990 (4h52m7.846734232s) [upload=5025]: Next returned nil segment Aug 7 21:18:54.259835595 (4h52m7.846891847s) [upload=5025]: Waiting for error group managing segments... Aug 7 21:18:54.269953672 (4h52m7.857009913s) [noise=true , address=45.91.116.7:4011 , piece_id=H2JRFAYC7KLC3TIXQ4PJZTKZLY552JJMZF6XTCFMHFD3RGIUMOCQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.270112598 (4h52m7.857168849s) [noise=true , address=45.91.116.8:4048 , piece_id=2IZI3WOTOBBKYHKVPJDQ6PGS5VTDAFIGTI373VWKT7QLAZR7W3FA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.270947997 (4h52m7.858004239s) [noise=true , address=45.91.116.5:4297 , piece_id=YK7QNB23S43MPEVNDLAT5WBFAMVBPQ3CVYKEMW5VBUHUAI5W4S5Q, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.271046852 (4h52m7.858103094s) [noise=true , address=77.90.31.13:4628 , piece_id=N3UNNLVQJ3GALE77VUERGE4PJDIYJK4T5VJEARML2QSMNJ2AAPJA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.272223499 (4h52m7.85927975s) [noise=true , address=45.91.116.10:4083 , piece_id=HW5YZYWXC7XHQA37JIKXDA7SSO6HOVCCGDS2SHWP7ROPG6MHQ7ZA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.272541713 (4h52m7.859597954s) [noise=true , address=45.91.116.14:4189 , piece_id=GRDSWIHHZ4FEWBEHRY7WPEF546PJWGBIXXCYDESVV4N3GSAWVGYA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.273006260 (4h52m7.860062501s) [noise=true , address=45.91.116.8:4047 , piece_id=7F4MB4X3CJZH3DUXC7VXN3HGQBBRFUV62G4KLCKV7BSARABJENWQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.275575946 (4h52m7.862632188s) [noise=true , address=77.90.31.10:4520 , piece_id=Z5P2H3YFJDC5WMKMFIFVYGFVAU2R27KTNMIAYBDYZGRUT5TWN5QQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.281367938 (4h52m7.868424169s) [noise=true , address=77.90.31.15:4697 , piece_id=PILNZYYWJABT57ITSG77YXQZN7EJKSDHB75RJHE6SPB67FSWCPGQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.284511737 (4h52m7.871567998s) [noise=true , address=77.90.31.19:4841 , piece_id=AUK7XFVC5R73KNHLVVJVRGUTRWBEJ6LP4FZIUJ77DECJRXMHWRRA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.291770877 (4h52m7.878827127s) [noise=true , address=104.36.82.4:4803 , piece_id=DCMHD25G5K5S33FQ72K7VEQ3U6ZV5UBZK5UFVDWUI5I3BX6E4NHA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.298214294 (4h52m7.885270545s) [noise=true , address=77.90.31.11:4555 , piece_id=D7YHBQ32VTIJEC4HKWQYHWMOULL3KNN7P7TJSA7LV4TLUICYGWBQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.532849875 (4h52m8.119906117s) [noise=true , address=104.36.82.6:4876 , piece_id=6WRCDS47O3FUJLE2BT6QV3OOXKGONXLHZ7TMPXKHH4LP2AJGHV5Q, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.532867749 (4h52m8.11992399s) [seg_pos={0 0}, upload=5025]: Segment reached optimal threshold of 13 pieces. Aug 7 21:18:54.533278306 (4h52m8.120334548s) [noise=true , address=104.36.82.15:4259 , piece_id=ZNJ3QCHTB2LJVM7KI52E3RPGDYEHWWFTEEBBNUIOVPIDOQP7VTKQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Aug 7 21:18:54.533410502 (4h52m8.120466743s) [noise=true , address=104.36.82.15:4258 , piece_id=VT3VKJFUHTSDCNR7PIBIJXEDYG3HGIVT4JZRR5VM3WJ6A5RHQQZQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: expected piece hash; context canceled; EOF Aug 7 21:18:54.533558579 (4h52m8.12061482s) [noise=true , address=104.36.82.14:4223 , piece_id=LX33NZFO2TFV43ARCAIGFYXJZZRR633JWZTHSO6XWSRP7TQXU5HQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: write tcp 5.161.89.252:47372->104.36.82.14:4223: use of closed network connection; write tcp 5.161.89.252:47372->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47372->104.36.82.14:4223: use of closed network connection Aug 7 21:18:54.533575210 (4h52m8.120631451s) [seg_pos={0 0}, upload=5025]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:18:54.533666891 (4h52m8.120723132s) [upload=5025]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:18:54.533672582 (4h52m8.120728823s) [upload=5025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:20:54.197308723 (4h54m7.784364965s) [upload=5025]: Done uploading segments... Aug 7 21:20:54.197331416 (4h54m7.784387657s) [upload=5025]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:54.289617342 (4h54m7.876673583s) [upload=5062]: Starting upload Aug 7 21:20:54.289682233 (4h54m7.876738475s) [upload=5062]: Done starting upload Aug 7 21:20:54.290100865 (4h54m7.877157115s) [upload=5062]: Uploading segments... Aug 7 21:20:54.290145758 (4h54m7.877201999s) [upload=5062]: Got next segment. Inline: false Aug 7 21:20:54.290178920 (4h54m7.877235161s) [upload=5062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:54.290184501 (4h54m7.877240741s) [upload=5062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:21:24.274797304 (4h54m37.861853546s) [upload=5062]: Next segment err: context deadline exceeded Aug 7 21:21:24.274996005 (4h54m37.862052246s) [upload=5062]: Done uploading segments... Aug 7 21:21:24.275019569 (4h54m37.86207581s) [upload=5062]: Upload finished. err: context deadline exceeded Aug 7 21:21:31.936040213 (4h54m45.523096464s) [upload=5073]: Starting upload Aug 7 21:21:31.936133387 (4h54m45.523189638s) [upload=5073]: Done starting upload Aug 7 21:21:31.936256627 (4h54m45.523312868s) [upload=5073]: Uploading segments... Aug 7 21:21:31.936495583 (4h54m45.523551824s) [upload=5073]: Got next segment. Inline: false Aug 7 21:21:31.936532813 (4h54m45.523589053s) [upload=5073]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:31.936537992 (4h54m45.523594234s) [upload=5073]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:22:01.921915749 (4h55m15.50897196s) [upload=5073]: Next segment err: context deadline exceeded Aug 7 21:22:01.922047263 (4h55m15.509103475s) [upload=5073]: Done uploading segments... Aug 7 21:22:01.922079755 (4h55m15.509135976s) [upload=5073]: Upload finished. err: context deadline exceeded Duration: 30.000593689s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.................................................XXXXXXXXXXX] duration: 30.000483411s FAIL (2025-08-07 21:21:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000498718s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 9 out of 60 is failed (threshold=3) [..................................................XX.XXXXXXX] duration: 1m0.000655162s FAIL (2025-08-07 21:21:51 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000670159s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.....................................................XXXXXXX] duration: 30.001096175s FAIL (2025-08-07 21:21:46 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.001122594s core uplink us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 37.509769ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 21:21:42 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 37.541528ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 7 out of 60 is failed (threshold=3) [.....................................................XXXXXXX] duration: 1m10.474867123s FAIL (2025-08-07 21:21:40 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== (4h53m43.594808058s) [upload=5054]: Done starting upload Aug 7 21:20:30.007805085 (4h53m43.594861306s) [upload=5054]: Uploading segments... Aug 7 21:20:30.008037719 (4h53m43.59509394s) [upload=5054]: Got next segment. Inline: false Aug 7 21:20:30.008083926 (4h53m43.595140137s) [upload=5054]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:30.008088305 (4h53m43.595144515s) [upload=5054]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:21:01.629736413 (4h54m15.216792634s) [seg_pos={0 0}, upload=5054]: Begin upload segment... Aug 7 21:21:01.629774765 (4h54m15.216830975s) [seg_pos={0 0}, upload=5054]: Done begin upload segment. Aug 7 21:21:01.629801183 (4h54m15.216857404s) [noise=true , address=104.36.82.45:4927 , piece_id=XOJ54RWLE5XHICCTBCACP54GD4PBZTP37IG57DX4BQVV7CUOU2KA, seg_pos={0 0}, upload=5054]: Uploading piece... Aug 7 21:21:01.629808237 (4h54m15.216864458s) [noise=true , address=104.36.82.5:4869 , piece_id=TJ2FEL2IU32MLMEY2YXRWRJXPPWMCOK3W5FOHZ7IKWKUPKBBZNJQ, seg_pos={0 0}, upload=5054]: Uploading piece... Aug 7 21:21:01.629844725 (4h54m15.216900936s) [noise=true , address=64.49.13.137:4500 , piece_id=PTZLYH5OQRWKIQU4YEWD24OWLG6ODXDFISK7MNZVOMX4YYQPOASQ, seg_pos={0 0}, upload=5054]: Uploading piece... Aug 7 21:21:01.630100663 (4h54m15.217156874s) [noise=true , address=104.36.82.11:4121 , piece_id=46LUZFNPPPEL6TH4IVTSKGLXHVVACNRBD2JF4EQKJUXAM5JGS2UA, seg_pos={0 0}, upload=5054]: Uploading piece... Aug 7 21:21:01.630104971 (4h54m15.217161192s) [noise=true , address=104.36.82.34:4548 , piece_id=CJNMTOURRPWIRG6NZUOYSRDOCVAVKLIIR3EDBL3RCY6NPOK7J6NQ, seg_pos={0 0}, upload=5054]: Uploading piece... Aug 7 21:21:01.630325743 (4h54m15.217381954s) [noise=true , address=104.36.82.36:4638 , piece_id=UNUQYP2A23Y7A2MUFBH2LDJQAK4WKAZEO7I2F5AD6J4KEFGWSAWA, seg_pos={0 0}, upload=5054]: Uploading piece... Aug 7 21:21:01.630334009 (4h54m15.21739023s) [noise=true , address=104.36.82.18:4400 , piece_id=E5AQRTJJON7ORYASAPTLQ4M33YBCOUYGVMI2BW44BQ6GRJCH5NRQ, seg_pos={0 0}, upload=5054]: Uploading piece... Aug 7 21:21:01.630542026 (4h54m15.217598238s) [noise=true , address=104.36.82.59:4328 , piece_id=SHHVUETGPXF32CVWNPBGB7ZUMOZTDYUGVWSMJ4Z56GEREMQEYESA, seg_pos={0 0}, upload=5054]: Uploading piece... Aug 7 21:21:01.630546475 (4h54m15.217602686s) [noise=true , address=104.36.82.46:4496 , piece_id=B3UO6PUTGP7IOEHLPEZYM2X57MTV6DRKOY5QVJ4LC3RUWHJ6PGQQ, seg_pos={0 0}, upload=5054]: Uploading piece... Aug 7 21:21:01.630745686 (4h54m15.217801907s) [noise=true , address=104.36.82.39:4729 , piece_id=USKXKJAKLC527FRPZECZ3EBHR4WVCOITAKZVUBAC5OS2UCPG3DHA, seg_pos={0 0}, upload=5054]: Uploading piece... Aug 7 21:21:01.630753902 (4h54m15.217810113s) [noise=true , address=104.36.82.46:4503 , piece_id=NALQAIFZM77H3ETGK3DFGPNTITZOTVW27726MJ66EWBZRNZLOWTA, seg_pos={0 0}, upload=5054]: Uploading piece... Aug 7 21:21:01.631025269 (4h54m15.21808148s) [noise=true , address=104.36.82.35:4603 , piece_id=RM5SA63BORFOVAYHWFYA5RIBVAMDHG7342N6STYZEQBCGA4VRUPQ, seg_pos={0 0}, upload=5054]: Uploading piece... Aug 7 21:21:01.631275536 (4h54m15.218331747s) [noise=true , address=104.36.82.23:4448 , piece_id=2CXHBVFJ3LJ2CD45RJECLSS2CRZCGLZJUHD7I6Z5I4CF375JIOFQ, seg_pos={0 0}, upload=5054]: Uploading piece... Aug 7 21:21:01.631509182 (4h54m15.218565403s) [noise=true , address=79.127.205.233:4636 , piece_id=HHDOHB3ODMICIJJM3YD7S4B6ZAHWJ7VSQW53PNV4T2XZVS6R7Y3Q, seg_pos={0 0}, upload=5054]: Uploading piece... Aug 7 21:21:01.631790597 (4h54m15.218846808s) [noise=true , address=104.36.82.52:4090 , piece_id=2RH5KCTQKBTMVWPLQIHZZMKTWBYJ4RNVN77SEMGOLOS5LRYGMBOQ, seg_pos={0 0}, upload=5054]: Uploading piece... Aug 7 21:21:01.632025195 (4h54m15.219081416s) [noise=true , address=104.36.82.8:4978 , piece_id=RPLYYKXS2HKXK2PYFSDKW432TGU6EAU32E64WMU3ZIRTZLFW6URQ, seg_pos={0 0}, upload=5054]: Uploading piece... Aug 7 21:21:01.632913945 (4h54m15.219970166s) [noise=true , address=45.91.116.9:4286 , piece_id=ADYBKWFYE7S7PLBXXJSEZL6EDPNACGOCWPJUUYYGO3JVBPFK3XVQ, seg_pos={0 0}, upload=5054]: Uploading piece... Aug 7 21:21:01.634201908 (4h54m15.221258129s) [noise=true , address=104.36.82.15:4281 , piece_id=6TOCJUUM4WHOIAMMB47E2T42RHMEEOPNSZFCLUBUGM3MTGOKZNUA, seg_pos={0 0}, upload=5054]: Uploading piece... Aug 7 21:21:01.699770281 (4h54m15.286826492s) [upload=5054]: Next returned nil segment Aug 7 21:21:01.699782264 (4h54m15.286838475s) [upload=5054]: Waiting for error group managing segments... Aug 7 21:21:01.712486898 (4h54m15.29954311s) [noise=true , address=45.91.116.9:4286 , piece_id=ADYBKWFYE7S7PLBXXJSEZL6EDPNACGOCWPJUUYYGO3JVBPFK3XVQ, seg_pos={0 0}, upload=5054]: Done uploading piece. err: <nil> Aug 7 21:21:01.733155479 (4h54m15.3202117s) [noise=true , address=104.36.82.11:4121 , piece_id=46LUZFNPPPEL6TH4IVTSKGLXHVVACNRBD2JF4EQKJUXAM5JGS2UA, seg_pos={0 0}, upload=5054]: Done uploading piece. err: <nil> Aug 7 21:21:01.741807158 (4h54m15.328863369s) [noise=true , address=104.36.82.45:4927 , piece_id=XOJ54RWLE5XHICCTBCACP54GD4PBZTP37IG57DX4BQVV7CUOU2KA, seg_pos={0 0}, upload=5054]: Done uploading piece. err: <nil> Aug 7 21:21:01.744132189 (4h54m15.33118841s) [noise=true , address=104.36.82.35:4603 , piece_id=RM5SA63BORFOVAYHWFYA5RIBVAMDHG7342N6STYZEQBCGA4VRUPQ, seg_pos={0 0}, upload=5054]: Done uploading piece. err: <nil> Aug 7 21:21:01.750460823 (4h54m15.337517034s) [noise=true , address=104.36.82.8:4978 , piece_id=RPLYYKXS2HKXK2PYFSDKW432TGU6EAU32E64WMU3ZIRTZLFW6URQ, seg_pos={0 0}, upload=5054]: Done uploading piece. err: <nil> Aug 7 21:21:01.750663782 (4h54m15.337720012s) [noise=true , address=104.36.82.34:4548 , piece_id=CJNMTOURRPWIRG6NZUOYSRDOCVAVKLIIR3EDBL3RCY6NPOK7J6NQ, seg_pos={0 0}, upload=5054]: Done uploading piece. err: <nil> Aug 7 21:21:01.751690488 (4h54m15.338746699s) [noise=true , address=104.36.82.59:4328 , piece_id=SHHVUETGPXF32CVWNPBGB7ZUMOZTDYUGVWSMJ4Z56GEREMQEYESA, seg_pos={0 0}, upload=5054]: Done uploading piece. err: <nil> Aug 7 21:21:01.753403074 (4h54m15.340459295s) [noise=true , address=104.36.82.36:4638 , piece_id=UNUQYP2A23Y7A2MUFBH2LDJQAK4WKAZEO7I2F5AD6J4KEFGWSAWA, seg_pos={0 0}, upload=5054]: Done uploading piece. err: <nil> Aug 7 21:21:01.753967117 (4h54m15.341023328s) [noise=true , address=104.36.82.52:4090 , piece_id=2RH5KCTQKBTMVWPLQIHZZMKTWBYJ4RNVN77SEMGOLOS5LRYGMBOQ, seg_pos={0 0}, upload=5054]: Done uploading piece. err: <nil> Aug 7 21:21:01.755108558 (4h54m15.342164779s) [noise=true , address=104.36.82.23:4448 , piece_id=2CXHBVFJ3LJ2CD45RJECLSS2CRZCGLZJUHD7I6Z5I4CF375JIOFQ, seg_pos={0 0}, upload=5054]: Done uploading piece. err: <nil> Aug 7 21:21:01.766576384 (4h54m15.353632605s) [noise=true , address=104.36.82.18:4400 , piece_id=E5AQRTJJON7ORYASAPTLQ4M33YBCOUYGVMI2BW44BQ6GRJCH5NRQ, seg_pos={0 0}, upload=5054]: Done uploading piece. err: <nil> Aug 7 21:21:01.767762108 (4h54m15.354818329s) [noise=true , address=104.36.82.39:4729 , piece_id=USKXKJAKLC527FRPZECZ3EBHR4WVCOITAKZVUBAC5OS2UCPG3DHA, seg_pos={0 0}, upload=5054]: Done uploading piece. err: <nil> Aug 7 21:21:01.770716843 (4h54m15.357773064s) [noise=true , address=104.36.82.5:4869 , piece_id=TJ2FEL2IU32MLMEY2YXRWRJXPPWMCOK3W5FOHZ7IKWKUPKBBZNJQ, seg_pos={0 0}, upload=5054]: Done uploading piece. err: <nil> Aug 7 21:21:01.799326808 (4h54m15.386383029s) [noise=true , address=104.36.82.15:4281 , piece_id=6TOCJUUM4WHOIAMMB47E2T42RHMEEOPNSZFCLUBUGM3MTGOKZNUA, seg_pos={0 0}, upload=5054]: Done uploading piece. err: <nil> Aug 7 21:21:01.799365411 (4h54m15.386421632s) [seg_pos={0 0}, upload=5054]: Segment reached optimal threshold of 14 pieces. Aug 7 21:21:01.799553221 (4h54m15.386609432s) [noise=true , address=104.36.82.46:4503 , piece_id=NALQAIFZM77H3ETGK3DFGPNTITZOTVW27726MJ66EWBZRNZLOWTA, seg_pos={0 0}, upload=5054]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ctmSF1aMQasuXByFW8DppDantREmmWL2WUjmKGW5ZULVePyvm): protocol: expected piece hash; context canceled; EOF Aug 7 21:21:01.799574861 (4h54m15.386631082s) [noise=true , address=104.36.82.46:4496 , piece_id=B3UO6PUTGP7IOEHLPEZYM2X57MTV6DRKOY5QVJ4LC3RUWHJ6PGQQ, seg_pos={0 0}, upload=5054]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF Aug 7 21:21:01.799813867 (4h54m15.386870079s) [noise=true , address=79.127.205.233:4636 , piece_id=HHDOHB3ODMICIJJM3YD7S4B6ZAHWJ7VSQW53PNV4T2XZVS6R7Y3Q, seg_pos={0 0}, upload=5054]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mTJmjiHwMJqSkuWKrFnAgJUt1gjQ4wPjfgY7qouWcanaNUy46): protocol: read tcp 5.161.89.252:35886->79.127.205.233:4636: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:21:01.799876174 (4h54m15.386932395s) [noise=true , address=64.49.13.137:4500 , piece_id=PTZLYH5OQRWKIQU4YEWD24OWLG6ODXDFISK7MNZVOMX4YYQPOASQ, seg_pos={0 0}, upload=5054]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15TwVQ6CHDbtqS4XW86n9BuyjdqpaJo4ZKPgPnmhSm6EPDrnd9): protocol: read tcp 5.161.89.252:43024->64.49.13.137:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:21:01.799936918 (4h54m15.386993139s) [seg_pos={0 0}, upload=5054]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:21:01.800028308 (4h54m15.387084519s) [upload=5054]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:21:01.800033077 (4h54m15.387089298s) [upload=5054]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:21:40.466597494 (4h54m54.053653736s) [upload=5054]: Done uploading segments... Aug 7 21:21:40.466760558 (4h54m54.0538168s) [upload=5054]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m10.474890197s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 874.61147ms FAIL (2025-08-07 21:21:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 874.631649ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 11 out of 60 is failed (threshold=3) [.................................................XXXXXXXXXXX] duration: 30.001569856s FAIL (2025-08-07 21:21:33 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== xceeded Aug 7 21:13:15.702096720 (4h46m29.289152941s) [upload=4939]: Starting upload Aug 7 21:13:15.702154518 (4h46m29.289210739s) [upload=4939]: Done starting upload Aug 7 21:13:15.702196716 (4h46m29.289252928s) [upload=4939]: Uploading segments... Aug 7 21:13:15.702224368 (4h46m29.289280589s) [upload=4939]: Got next segment. Inline: true Aug 7 21:13:15.702228446 (4h46m29.289284657s) [upload=4939]: Waiting for error group managing segments... Aug 7 21:13:15.702261458 (4h46m29.289317669s) [upload=4939]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:13:15.702265306 (4h46m29.289321527s) [upload=4939]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:13:15.702267880 (4h46m29.289324101s) [upload=4939]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:13:45.689162551 (4h46m59.276218792s) [upload=4939]: Done uploading segments... Aug 7 21:13:45.689208046 (4h46m59.276264277s) [upload=4939]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:14:13.522007492 (4h47m27.109063713s) [upload=4955]: Starting upload Aug 7 21:14:13.522102209 (4h47m27.10915843s) [upload=4955]: Done starting upload Aug 7 21:14:13.522154938 (4h47m27.109211179s) [upload=4955]: Uploading segments... Aug 7 21:14:13.522235488 (4h47m27.109291729s) [upload=4955]: Got next segment. Inline: true Aug 7 21:14:13.522242682 (4h47m27.109298922s) [upload=4955]: Waiting for error group managing segments... Aug 7 21:14:13.522293467 (4h47m27.109349708s) [upload=4955]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:13.522299518 (4h47m27.109355759s) [upload=4955]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:14:13.522303275 (4h47m27.109359516s) [upload=4955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:14:43.507883903 (4h47m57.094940144s) [upload=4955]: Done uploading segments... Aug 7 21:14:43.507905914 (4h47m57.094962156s) [upload=4955]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:14:54.825501522 (4h48m8.412557734s) [upload=4964]: Starting upload Aug 7 21:14:54.825580730 (4h48m8.412636951s) [upload=4964]: Done starting upload Aug 7 21:14:54.825629411 (4h48m8.412685632s) [upload=4964]: Uploading segments... Aug 7 21:14:54.825660400 (4h48m8.41271661s) [upload=4964]: Got next segment. Inline: true Aug 7 21:14:54.825664547 (4h48m8.412720758s) [upload=4964]: Waiting for error group managing segments... Aug 7 21:14:54.825705873 (4h48m8.412762115s) [upload=4964]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:54.825711855 (4h48m8.412768076s) [upload=4964]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:14:54.825715662 (4h48m8.412771883s) [upload=4964]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:15:24.812961328 (4h48m38.400017569s) [upload=4964]: Done uploading segments... Aug 7 21:15:24.812988770 (4h48m38.40004501s) [upload=4964]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:15:40.993035983 (4h48m54.580092194s) [upload=4978]: Starting upload Aug 7 21:15:40.993105403 (4h48m54.580161614s) [upload=4978]: Done starting upload Aug 7 21:15:40.993154163 (4h48m54.580210375s) [upload=4978]: Uploading segments... Aug 7 21:15:40.993189519 (4h48m54.580245741s) [upload=4978]: Got next segment. Inline: true Aug 7 21:15:40.993194319 (4h48m54.58025054s) [upload=4978]: Waiting for error group managing segments... Aug 7 21:15:40.993226930 (4h48m54.580283151s) [upload=4978]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:15:40.993231167 (4h48m54.580287379s) [upload=4978]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:40.993234264 (4h48m54.580290474s) [upload=4978]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:16:10.982081529 (4h49m24.569137771s) [upload=4978]: Done uploading segments... Aug 7 21:16:10.982104473 (4h49m24.569160713s) [upload=4978]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:20.227399180 (4h49m33.814455401s) [upload=4990]: Starting upload Aug 7 21:16:20.227450334 (4h49m33.814506545s) [upload=4990]: Done starting upload Aug 7 21:16:20.227491060 (4h49m33.814547272s) [upload=4990]: Uploading segments... Aug 7 21:16:20.227516609 (4h49m33.81457283s) [upload=4990]: Got next segment. Inline: true Aug 7 21:16:20.227520536 (4h49m33.814576757s) [upload=4990]: Waiting for error group managing segments... Aug 7 21:16:20.227552426 (4h49m33.814608636s) [upload=4990]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:16:20.227556263 (4h49m33.814612484s) [upload=4990]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:20.227558527 (4h49m33.814614739s) [upload=4990]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:16:50.216081431 (4h50m3.803137652s) [upload=4990]: Done uploading segments... Aug 7 21:16:50.216097492 (4h50m3.803153713s) [upload=4990]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:57.322533914 (4h50m10.909590156s) [upload=4995]: Starting upload Aug 7 21:16:57.322609355 (4h50m10.909665607s) [upload=4995]: Done starting upload Aug 7 21:16:57.322672883 (4h50m10.909729125s) [upload=4995]: Uploading segments... Aug 7 21:16:57.322711135 (4h50m10.909767376s) [upload=4995]: Got next segment. Inline: true Aug 7 21:16:57.322716685 (4h50m10.909772927s) [upload=4995]: Waiting for error group managing segments... Aug 7 21:16:57.322761279 (4h50m10.90981752s) [upload=4995]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:16:57.322767100 (4h50m10.909823341s) [upload=4995]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:57.322770697 (4h50m10.909826938s) [upload=4995]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:17:27.310589172 (4h50m40.897645383s) [upload=4995]: Done uploading segments... Aug 7 21:17:27.310603289 (4h50m40.897659499s) [upload=4995]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:38.458031408 (4h50m52.045087639s) [upload=5010]: Starting upload Aug 7 21:17:38.458135221 (4h50m52.045191452s) [upload=5010]: Done starting upload Aug 7 21:17:38.458199552 (4h50m52.045255784s) [upload=5010]: Uploading segments... Aug 7 21:17:38.458275263 (4h50m52.045331504s) [upload=5010]: Got next segment. Inline: true Aug 7 21:17:38.458283519 (4h50m52.04533976s) [upload=5010]: Waiting for error group managing segments... Aug 7 21:17:38.458329695 (4h50m52.045385926s) [upload=5010]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:17:38.458335165 (4h50m52.045391407s) [upload=5010]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:38.458338662 (4h50m52.045394893s) [upload=5010]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:18:08.444937554 (4h51m22.031993765s) [upload=5010]: Done uploading segments... Aug 7 21:18:08.444950428 (4h51m22.032006639s) [upload=5010]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:50.205377240 (4h52m3.792433451s) [upload=5028]: Starting upload Aug 7 21:18:50.205432072 (4h52m3.792488293s) [upload=5028]: Done starting upload Aug 7 21:18:50.205472938 (4h52m3.792529149s) [upload=5028]: Uploading segments... Aug 7 21:18:50.205500099 (4h52m3.792556311s) [upload=5028]: Got next segment. Inline: true Aug 7 21:18:50.205503937 (4h52m3.792560147s) [upload=5028]: Waiting for error group managing segments... Aug 7 21:18:50.205531809 (4h52m3.79258802s) [upload=5028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:18:50.205535356 (4h52m3.792591577s) [upload=5028]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:50.205538692 (4h52m3.792594913s) [upload=5028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:19:20.193730995 (4h52m33.780787246s) [upload=5028]: Done uploading segments... Aug 7 21:19:20.193750121 (4h52m33.780806372s) [upload=5028]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:03.998495850 (4h53m17.58555209s) [upload=5048]: Starting upload Aug 7 21:20:03.998559819 (4h53m17.585616059s) [upload=5048]: Done starting upload Aug 7 21:20:03.998621224 (4h53m17.585677465s) [upload=5048]: Uploading segments... Aug 7 21:20:03.998661639 (4h53m17.585717879s) [upload=5048]: Got next segment. Inline: true Aug 7 21:20:03.998667419 (4h53m17.58572365s) [upload=5048]: Waiting for error group managing segments... Aug 7 21:20:03.998710790 (4h53m17.585767032s) [upload=5048]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:03.998716000 (4h53m17.585772242s) [upload=5048]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:20:03.998719587 (4h53m17.585775818s) [upload=5048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:20:33.984947930 (4h53m47.572004141s) [upload=5048]: Done uploading segments... Aug 7 21:20:33.985025244 (4h53m47.572081455s) [upload=5048]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:03.692382456 (4h54m17.279438677s) [upload=5064]: Starting upload Aug 7 21:21:03.692428231 (4h54m17.279484453s) [upload=5064]: Done starting upload Aug 7 21:21:03.692467736 (4h54m17.279523957s) [upload=5064]: Uploading segments... Aug 7 21:21:03.692495146 (4h54m17.279551367s) [upload=5064]: Got next segment. Inline: true Aug 7 21:21:03.692500066 (4h54m17.279556287s) [upload=5064]: Waiting for error group managing segments... Aug 7 21:21:03.692529661 (4h54m17.279585882s) [upload=5064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:03.692533137 (4h54m17.279589349s) [upload=5064]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:03.692535481 (4h54m17.279591702s) [upload=5064]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:21:33.679184065 (4h54m47.266240287s) [upload=5064]: Done uploading segments... Aug 7 21:21:33.679200356 (4h54m47.266256568s) [upload=5064]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.001612045s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 9 out of 60 is failed (threshold=3) [...................................................XXXXXXXXX] duration: 30.000677944s FAIL (2025-08-07 21:21:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000690237s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 7 out of 60 is failed (threshold=3) [.....................................................XXXXXXX] duration: 1m0.000778877s FAIL (2025-08-07 21:21:30 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ading segments... Aug 7 21:11:31.682853265 (4h44m45.269909476s) [upload=4908]: Upload finished. err: <nil> Aug 7 21:11:51.754258397 (4h45m5.341314618s) [upload=4915]: Starting upload Aug 7 21:11:51.754330331 (4h45m5.341386542s) [upload=4915]: Done starting upload Aug 7 21:11:51.754383972 (4h45m5.341440193s) [upload=4915]: Uploading segments... Aug 7 21:11:51.754417083 (4h45m5.341473294s) [upload=4915]: Got next segment. Inline: true Aug 7 21:11:51.754420630 (4h45m5.34147684s) [upload=4915]: Waiting for error group managing segments... Aug 7 21:11:51.754453131 (4h45m5.341509352s) [upload=4915]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:11:51.754457108 (4h45m5.341513329s) [upload=4915]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:11:51.754460725 (4h45m5.341516936s) [upload=4915]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:11:51.830658438 (4h45m5.417714649s) [upload=4915]: Done uploading segments... Aug 7 21:11:51.830673094 (4h45m5.417729315s) [upload=4915]: Upload finished. err: <nil> Aug 7 21:12:54.778494646 (4h46m8.365550856s) [upload=4932]: Starting upload Aug 7 21:12:54.778560719 (4h46m8.36561693s) [upload=4932]: Done starting upload Aug 7 21:12:54.778608638 (4h46m8.365664859s) [upload=4932]: Uploading segments... Aug 7 21:12:54.778637322 (4h46m8.365693543s) [upload=4932]: Got next segment. Inline: true Aug 7 21:12:54.778640999 (4h46m8.36569722s) [upload=4932]: Waiting for error group managing segments... Aug 7 21:12:54.778677527 (4h46m8.365733739s) [upload=4932]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:12:54.778681655 (4h46m8.365737876s) [upload=4932]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:12:54.778684771 (4h46m8.365740983s) [upload=4932]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:12:54.868239378 (4h46m8.455295599s) [upload=4932]: Done uploading segments... Aug 7 21:12:54.868253865 (4h46m8.455310086s) [upload=4932]: Upload finished. err: <nil> Aug 7 21:13:23.238906839 (4h46m36.82596306s) [upload=4942]: Starting upload Aug 7 21:13:23.238959356 (4h46m36.826015567s) [upload=4942]: Done starting upload Aug 7 21:13:23.239011013 (4h46m36.826067234s) [upload=4942]: Uploading segments... Aug 7 21:13:23.239039727 (4h46m36.826095937s) [upload=4942]: Got next segment. Inline: true Aug 7 21:13:23.239043483 (4h46m36.826099695s) [upload=4942]: Waiting for error group managing segments... Aug 7 21:13:23.239098526 (4h46m36.826154737s) [upload=4942]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:13:23.239104707 (4h46m36.826160919s) [upload=4942]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:13:23.239108474 (4h46m36.826164696s) [upload=4942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:13:23.309357825 (4h46m36.896414127s) [upload=4942]: Done uploading segments... Aug 7 21:13:23.309383974 (4h46m36.896440216s) [upload=4942]: Upload finished. err: <nil> Aug 7 21:14:18.294119361 (4h47m31.881175572s) [upload=4957]: Starting upload Aug 7 21:14:18.294187468 (4h47m31.881243679s) [upload=4957]: Done starting upload Aug 7 21:14:18.294261516 (4h47m31.881317737s) [upload=4957]: Uploading segments... Aug 7 21:14:18.294305068 (4h47m31.881361289s) [upload=4957]: Got next segment. Inline: true Aug 7 21:14:18.294311409 (4h47m31.881367621s) [upload=4957]: Waiting for error group managing segments... Aug 7 21:14:18.294349491 (4h47m31.881405712s) [upload=4957]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:18.294354059 (4h47m31.88141028s) [upload=4957]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:14:18.294356643 (4h47m31.881412854s) [upload=4957]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:15:18.280461255 (4h48m31.867517496s) [upload=4957]: Done uploading segments... Aug 7 21:15:18.280484308 (4h48m31.867540559s) [upload=4957]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:15:18.387536247 (4h48m31.974592488s) [upload=4974]: Starting upload Aug 7 21:15:18.387607770 (4h48m31.974664011s) [upload=4974]: Done starting upload Aug 7 21:15:18.387666109 (4h48m31.974722361s) [upload=4974]: Uploading segments... Aug 7 21:15:18.387706996 (4h48m31.974763236s) [upload=4974]: Got next segment. Inline: true Aug 7 21:15:18.387712766 (4h48m31.974769007s) [upload=4974]: Waiting for error group managing segments... Aug 7 21:15:18.387755796 (4h48m31.974812038s) [upload=4974]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:15:18.387761227 (4h48m31.974817468s) [upload=4974]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:18.387764853 (4h48m31.974821095s) [upload=4974]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:16:18.373970870 (4h49m31.961027121s) [upload=4974]: Done uploading segments... Aug 7 21:16:18.374035110 (4h49m31.961091352s) [upload=4974]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:16:18.454541239 (4h49m32.04159748s) [upload=4989]: Starting upload Aug 7 21:16:18.454621288 (4h49m32.041677529s) [upload=4989]: Done starting upload Aug 7 21:16:18.454699754 (4h49m32.041756005s) [upload=4989]: Uploading segments... Aug 7 21:16:18.454758153 (4h49m32.041814395s) [upload=4989]: Got next segment. Inline: true Aug 7 21:16:18.454765147 (4h49m32.041821398s) [upload=4989]: Waiting for error group managing segments... Aug 7 21:16:18.454813777 (4h49m32.041870018s) [upload=4989]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:16:18.454819818 (4h49m32.04187606s) [upload=4989]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:16:18.454823425 (4h49m32.041879666s) [upload=4989]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:17:18.435319201 (4h50m32.022375413s) [upload=4989]: Done uploading segments... Aug 7 21:17:18.435335041 (4h50m32.022391262s) [upload=4989]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:17:29.913551414 (4h50m43.500607635s) [upload=5005]: Starting upload Aug 7 21:17:29.913606246 (4h50m43.500662467s) [upload=5005]: Done starting upload Aug 7 21:17:29.913653174 (4h50m43.500709395s) [upload=5005]: Uploading segments... Aug 7 21:17:29.913680986 (4h50m43.500737207s) [upload=5005]: Got next segment. Inline: true Aug 7 21:17:29.913684613 (4h50m43.500740835s) [upload=5005]: Waiting for error group managing segments... Aug 7 21:17:29.913715249 (4h50m43.500771471s) [upload=5005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:17:29.913719057 (4h50m43.500775278s) [upload=5005]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:29.913721331 (4h50m43.500777542s) [upload=5005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:18:29.900370534 (4h51m43.487426745s) [upload=5005]: Done uploading segments... Aug 7 21:18:29.900390291 (4h51m43.487446542s) [upload=5005]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:29.988006379 (4h51m43.5750626s) [upload=5023]: Starting upload Aug 7 21:18:29.988053607 (4h51m43.575109818s) [upload=5023]: Done starting upload Aug 7 21:18:29.988109782 (4h51m43.575166003s) [upload=5023]: Uploading segments... Aug 7 21:18:29.988142012 (4h51m43.575198224s) [upload=5023]: Got next segment. Inline: true Aug 7 21:18:29.988147753 (4h51m43.575203973s) [upload=5023]: Waiting for error group managing segments... Aug 7 21:18:29.988197475 (4h51m43.575253687s) [upload=5023]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:18:29.988203116 (4h51m43.575259327s) [upload=5023]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:29.988206943 (4h51m43.575263164s) [upload=5023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:19:29.967158594 (4h52m43.554214836s) [upload=5023]: Done uploading segments... Aug 7 21:19:29.967184642 (4h52m43.554240884s) [upload=5023]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:19:30.042411605 (4h52m43.629467826s) [upload=5040]: Starting upload Aug 7 21:19:30.042491373 (4h52m43.629547594s) [upload=5040]: Done starting upload Aug 7 21:19:30.042544984 (4h52m43.629601195s) [upload=5040]: Uploading segments... Aug 7 21:19:30.042585870 (4h52m43.629642091s) [upload=5040]: Got next segment. Inline: true Aug 7 21:19:30.042590809 (4h52m43.62964703s) [upload=5040]: Waiting for error group managing segments... Aug 7 21:19:30.042637226 (4h52m43.629693436s) [upload=5040]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:19:30.042642325 (4h52m43.629698546s) [upload=5040]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:19:30.042647144 (4h52m43.629703365s) [upload=5040]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:20:30.020861737 (4h53m43.607917958s) [upload=5040]: Done uploading segments... Aug 7 21:20:30.020878959 (4h53m43.60793518s) [upload=5040]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:30.129088028 (4h53m43.71614424s) [upload=5055]: Starting upload Aug 7 21:20:30.129144182 (4h53m43.716200404s) [upload=5055]: Done starting upload Aug 7 21:20:30.129191110 (4h53m43.716247332s) [upload=5055]: Uploading segments... Aug 7 21:20:30.129220234 (4h53m43.716276455s) [upload=5055]: Got next segment. Inline: true Aug 7 21:20:30.129223841 (4h53m43.716280062s) [upload=5055]: Waiting for error group managing segments... Aug 7 21:20:30.129266070 (4h53m43.716322282s) [upload=5055]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:30.129269998 (4h53m43.716326219s) [upload=5055]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:20:30.129272523 (4h53m43.716328734s) [upload=5055]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:21:30.117153252 (4h54m43.704209473s) [upload=5055]: Done uploading segments... Aug 7 21:21:30.117172058 (4h54m43.704228279s) [upload=5055]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.000791231s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.....................................................XXXXXXX] duration: 30.001390743s FAIL (2025-08-07 21:21:24 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== ]: Upload finished. err: context deadline exceeded Aug 7 21:18:36.577784924 (4h51m50.164841166s) [upload=5025]: Starting upload Aug 7 21:18:36.577862118 (4h51m50.164918349s) [upload=5025]: Done starting upload Aug 7 21:18:36.577953218 (4h51m50.165009459s) [upload=5025]: Uploading segments... Aug 7 21:18:36.578218493 (4h51m50.165274724s) [upload=5025]: Got next segment. Inline: false Aug 7 21:18:36.578268738 (4h51m50.165324978s) [upload=5025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:18:36.578284077 (4h51m50.165340317s) [upload=5025]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:18:54.119485924 (4h52m7.706542165s) [seg_pos={0 0}, upload=5025]: Begin upload segment... Aug 7 21:18:54.119532450 (4h52m7.706588691s) [seg_pos={0 0}, upload=5025]: Done begin upload segment. Aug 7 21:18:54.119561394 (4h52m7.706617645s) [noise=true , address=45.91.116.7:4011 , piece_id=H2JRFAYC7KLC3TIXQ4PJZTKZLY552JJMZF6XTCFMHFD3RGIUMOCQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.119574769 (4h52m7.70663101s) [noise=true , address=45.91.116.8:4047 , piece_id=7F4MB4X3CJZH3DUXC7VXN3HGQBBRFUV62G4KLCKV7BSARABJENWQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.120057159 (4h52m7.7071134s) [noise=true , address=104.36.82.4:4803 , piece_id=DCMHD25G5K5S33FQ72K7VEQ3U6ZV5UBZK5UFVDWUI5I3BX6E4NHA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.120241273 (4h52m7.707297534s) [noise=true , address=104.36.82.6:4876 , piece_id=6WRCDS47O3FUJLE2BT6QV3OOXKGONXLHZ7TMPXKHH4LP2AJGHV5Q, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.120508351 (4h52m7.707564592s) [noise=true , address=104.36.82.15:4259 , piece_id=ZNJ3QCHTB2LJVM7KI52E3RPGDYEHWWFTEEBBNUIOVPIDOQP7VTKQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.120776011 (4h52m7.707832252s) [noise=true , address=77.90.31.15:4697 , piece_id=PILNZYYWJABT57ITSG77YXQZN7EJKSDHB75RJHE6SPB67FSWCPGQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.120880315 (4h52m7.707936557s) [noise=true , address=45.91.116.14:4189 , piece_id=GRDSWIHHZ4FEWBEHRY7WPEF546PJWGBIXXCYDESVV4N3GSAWVGYA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.121253463 (4h52m7.708309713s) [noise=true , address=45.91.116.10:4083 , piece_id=HW5YZYWXC7XHQA37JIKXDA7SSO6HOVCCGDS2SHWP7ROPG6MHQ7ZA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.121362616 (4h52m7.708418858s) [noise=true , address=77.90.31.10:4520 , piece_id=Z5P2H3YFJDC5WMKMFIFVYGFVAU2R27KTNMIAYBDYZGRUT5TWN5QQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.121608686 (4h52m7.708664927s) [noise=true , address=77.90.31.11:4555 , piece_id=D7YHBQ32VTIJEC4HKWQYHWMOULL3KNN7P7TJSA7LV4TLUICYGWBQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.121894870 (4h52m7.708951121s) [noise=true , address=45.91.116.8:4048 , piece_id=2IZI3WOTOBBKYHKVPJDQ6PGS5VTDAFIGTI373VWKT7QLAZR7W3FA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.121987422 (4h52m7.709043694s) [noise=true , address=77.90.31.19:4841 , piece_id=AUK7XFVC5R73KNHLVVJVRGUTRWBEJ6LP4FZIUJ77DECJRXMHWRRA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.122419388 (4h52m7.70947563s) [noise=true , address=104.36.82.15:4258 , piece_id=VT3VKJFUHTSDCNR7PIBIJXEDYG3HGIVT4JZRR5VM3WJ6A5RHQQZQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.122822722 (4h52m7.709878973s) [noise=true , address=77.90.31.13:4628 , piece_id=N3UNNLVQJ3GALE77VUERGE4PJDIYJK4T5VJEARML2QSMNJ2AAPJA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.123209864 (4h52m7.710266105s) [noise=true , address=45.91.116.5:4297 , piece_id=YK7QNB23S43MPEVNDLAT5WBFAMVBPQ3CVYKEMW5VBUHUAI5W4S5Q, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.123571199 (4h52m7.710627439s) [noise=true , address=104.36.82.14:4223 , piece_id=LX33NZFO2TFV43ARCAIGFYXJZZRR633JWZTHSO6XWSRP7TQXU5HQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.259677990 (4h52m7.846734232s) [upload=5025]: Next returned nil segment Aug 7 21:18:54.259835595 (4h52m7.846891847s) [upload=5025]: Waiting for error group managing segments... Aug 7 21:18:54.269953672 (4h52m7.857009913s) [noise=true , address=45.91.116.7:4011 , piece_id=H2JRFAYC7KLC3TIXQ4PJZTKZLY552JJMZF6XTCFMHFD3RGIUMOCQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.270112598 (4h52m7.857168849s) [noise=true , address=45.91.116.8:4048 , piece_id=2IZI3WOTOBBKYHKVPJDQ6PGS5VTDAFIGTI373VWKT7QLAZR7W3FA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.270947997 (4h52m7.858004239s) [noise=true , address=45.91.116.5:4297 , piece_id=YK7QNB23S43MPEVNDLAT5WBFAMVBPQ3CVYKEMW5VBUHUAI5W4S5Q, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.271046852 (4h52m7.858103094s) [noise=true , address=77.90.31.13:4628 , piece_id=N3UNNLVQJ3GALE77VUERGE4PJDIYJK4T5VJEARML2QSMNJ2AAPJA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.272223499 (4h52m7.85927975s) [noise=true , address=45.91.116.10:4083 , piece_id=HW5YZYWXC7XHQA37JIKXDA7SSO6HOVCCGDS2SHWP7ROPG6MHQ7ZA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.272541713 (4h52m7.859597954s) [noise=true , address=45.91.116.14:4189 , piece_id=GRDSWIHHZ4FEWBEHRY7WPEF546PJWGBIXXCYDESVV4N3GSAWVGYA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.273006260 (4h52m7.860062501s) [noise=true , address=45.91.116.8:4047 , piece_id=7F4MB4X3CJZH3DUXC7VXN3HGQBBRFUV62G4KLCKV7BSARABJENWQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.275575946 (4h52m7.862632188s) [noise=true , address=77.90.31.10:4520 , piece_id=Z5P2H3YFJDC5WMKMFIFVYGFVAU2R27KTNMIAYBDYZGRUT5TWN5QQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.281367938 (4h52m7.868424169s) [noise=true , address=77.90.31.15:4697 , piece_id=PILNZYYWJABT57ITSG77YXQZN7EJKSDHB75RJHE6SPB67FSWCPGQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.284511737 (4h52m7.871567998s) [noise=true , address=77.90.31.19:4841 , piece_id=AUK7XFVC5R73KNHLVVJVRGUTRWBEJ6LP4FZIUJ77DECJRXMHWRRA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.291770877 (4h52m7.878827127s) [noise=true , address=104.36.82.4:4803 , piece_id=DCMHD25G5K5S33FQ72K7VEQ3U6ZV5UBZK5UFVDWUI5I3BX6E4NHA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.298214294 (4h52m7.885270545s) [noise=true , address=77.90.31.11:4555 , piece_id=D7YHBQ32VTIJEC4HKWQYHWMOULL3KNN7P7TJSA7LV4TLUICYGWBQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.532849875 (4h52m8.119906117s) [noise=true , address=104.36.82.6:4876 , piece_id=6WRCDS47O3FUJLE2BT6QV3OOXKGONXLHZ7TMPXKHH4LP2AJGHV5Q, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.532867749 (4h52m8.11992399s) [seg_pos={0 0}, upload=5025]: Segment reached optimal threshold of 13 pieces. Aug 7 21:18:54.533278306 (4h52m8.120334548s) [noise=true , address=104.36.82.15:4259 , piece_id=ZNJ3QCHTB2LJVM7KI52E3RPGDYEHWWFTEEBBNUIOVPIDOQP7VTKQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Aug 7 21:18:54.533410502 (4h52m8.120466743s) [noise=true , address=104.36.82.15:4258 , piece_id=VT3VKJFUHTSDCNR7PIBIJXEDYG3HGIVT4JZRR5VM3WJ6A5RHQQZQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: expected piece hash; context canceled; EOF Aug 7 21:18:54.533558579 (4h52m8.12061482s) [noise=true , address=104.36.82.14:4223 , piece_id=LX33NZFO2TFV43ARCAIGFYXJZZRR633JWZTHSO6XWSRP7TQXU5HQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: write tcp 5.161.89.252:47372->104.36.82.14:4223: use of closed network connection; write tcp 5.161.89.252:47372->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47372->104.36.82.14:4223: use of closed network connection Aug 7 21:18:54.533575210 (4h52m8.120631451s) [seg_pos={0 0}, upload=5025]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:18:54.533666891 (4h52m8.120723132s) [upload=5025]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:18:54.533672582 (4h52m8.120728823s) [upload=5025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:20:54.197308723 (4h54m7.784364965s) [upload=5025]: Done uploading segments... Aug 7 21:20:54.197331416 (4h54m7.784387657s) [upload=5025]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:54.289617342 (4h54m7.876673583s) [upload=5062]: Starting upload Aug 7 21:20:54.289682233 (4h54m7.876738475s) [upload=5062]: Done starting upload Aug 7 21:20:54.290100865 (4h54m7.877157115s) [upload=5062]: Uploading segments... Aug 7 21:20:54.290145758 (4h54m7.877201999s) [upload=5062]: Got next segment. Inline: false Aug 7 21:20:54.290178920 (4h54m7.877235161s) [upload=5062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:54.290184501 (4h54m7.877240741s) [upload=5062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:21:24.274797304 (4h54m37.861853546s) [upload=5062]: Next segment err: context deadline exceeded Aug 7 21:21:24.274996005 (4h54m37.862052246s) [upload=5062]: Done uploading segments... Aug 7 21:21:24.275019569 (4h54m37.86207581s) [upload=5062]: Upload finished. err: context deadline exceeded Duration: 30.001428594s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..................................................XXXXXXXXXX] duration: 30.000889999s FAIL (2025-08-07 21:21:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000906059s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 1.148851663s FAIL (2025-08-07 21:21:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 1.14887671s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration: 30.00150349s FAIL (2025-08-07 21:21:08 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.001524901s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 8 out of 60 is failed (threshold=3) [....................................................XXXXXXXX] duration: 30.000888833s FAIL (2025-08-07 21:21:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000900013s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 12 out of 60 is failed (threshold=3) [..............................................XX..XXXXXXXXXX] duration: 30.000810992s FAIL (2025-08-07 21:20:58 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.000851998s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration: 2m17.63331805s FAIL (2025-08-07 21:20:54 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== oad finished. err: context deadline exceeded Aug 7 21:17:29.086779152 (4h50m42.673835373s) [upload=5004]: Starting upload Aug 7 21:17:29.086833233 (4h50m42.673889444s) [upload=5004]: Done starting upload Aug 7 21:17:29.086916819 (4h50m42.67397304s) [upload=5004]: Uploading segments... Aug 7 21:17:29.087068823 (4h50m42.674125034s) [upload=5004]: Got next segment. Inline: false Aug 7 21:17:29.087119998 (4h50m42.674176219s) [upload=5004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:17:29.087124536 (4h50m42.674180748s) [upload=5004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:17:59.072413634 (4h51m12.659469875s) [upload=5004]: Next segment err: context deadline exceeded Aug 7 21:17:59.072514252 (4h51m12.659570503s) [upload=5004]: Done uploading segments... Aug 7 21:17:59.072534319 (4h51m12.659590571s) [upload=5004]: Upload finished. err: context deadline exceeded Aug 7 21:18:36.577784924 (4h51m50.164841166s) [upload=5025]: Starting upload Aug 7 21:18:36.577862118 (4h51m50.164918349s) [upload=5025]: Done starting upload Aug 7 21:18:36.577953218 (4h51m50.165009459s) [upload=5025]: Uploading segments... Aug 7 21:18:36.578218493 (4h51m50.165274724s) [upload=5025]: Got next segment. Inline: false Aug 7 21:18:36.578268738 (4h51m50.165324978s) [upload=5025]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:18:36.578284077 (4h51m50.165340317s) [upload=5025]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:18:54.119485924 (4h52m7.706542165s) [seg_pos={0 0}, upload=5025]: Begin upload segment... Aug 7 21:18:54.119532450 (4h52m7.706588691s) [seg_pos={0 0}, upload=5025]: Done begin upload segment. Aug 7 21:18:54.119561394 (4h52m7.706617645s) [noise=true , address=45.91.116.7:4011 , piece_id=H2JRFAYC7KLC3TIXQ4PJZTKZLY552JJMZF6XTCFMHFD3RGIUMOCQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.119574769 (4h52m7.70663101s) [noise=true , address=45.91.116.8:4047 , piece_id=7F4MB4X3CJZH3DUXC7VXN3HGQBBRFUV62G4KLCKV7BSARABJENWQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.120057159 (4h52m7.7071134s) [noise=true , address=104.36.82.4:4803 , piece_id=DCMHD25G5K5S33FQ72K7VEQ3U6ZV5UBZK5UFVDWUI5I3BX6E4NHA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.120241273 (4h52m7.707297534s) [noise=true , address=104.36.82.6:4876 , piece_id=6WRCDS47O3FUJLE2BT6QV3OOXKGONXLHZ7TMPXKHH4LP2AJGHV5Q, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.120508351 (4h52m7.707564592s) [noise=true , address=104.36.82.15:4259 , piece_id=ZNJ3QCHTB2LJVM7KI52E3RPGDYEHWWFTEEBBNUIOVPIDOQP7VTKQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.120776011 (4h52m7.707832252s) [noise=true , address=77.90.31.15:4697 , piece_id=PILNZYYWJABT57ITSG77YXQZN7EJKSDHB75RJHE6SPB67FSWCPGQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.120880315 (4h52m7.707936557s) [noise=true , address=45.91.116.14:4189 , piece_id=GRDSWIHHZ4FEWBEHRY7WPEF546PJWGBIXXCYDESVV4N3GSAWVGYA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.121253463 (4h52m7.708309713s) [noise=true , address=45.91.116.10:4083 , piece_id=HW5YZYWXC7XHQA37JIKXDA7SSO6HOVCCGDS2SHWP7ROPG6MHQ7ZA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.121362616 (4h52m7.708418858s) [noise=true , address=77.90.31.10:4520 , piece_id=Z5P2H3YFJDC5WMKMFIFVYGFVAU2R27KTNMIAYBDYZGRUT5TWN5QQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.121608686 (4h52m7.708664927s) [noise=true , address=77.90.31.11:4555 , piece_id=D7YHBQ32VTIJEC4HKWQYHWMOULL3KNN7P7TJSA7LV4TLUICYGWBQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.121894870 (4h52m7.708951121s) [noise=true , address=45.91.116.8:4048 , piece_id=2IZI3WOTOBBKYHKVPJDQ6PGS5VTDAFIGTI373VWKT7QLAZR7W3FA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.121987422 (4h52m7.709043694s) [noise=true , address=77.90.31.19:4841 , piece_id=AUK7XFVC5R73KNHLVVJVRGUTRWBEJ6LP4FZIUJ77DECJRXMHWRRA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.122419388 (4h52m7.70947563s) [noise=true , address=104.36.82.15:4258 , piece_id=VT3VKJFUHTSDCNR7PIBIJXEDYG3HGIVT4JZRR5VM3WJ6A5RHQQZQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.122822722 (4h52m7.709878973s) [noise=true , address=77.90.31.13:4628 , piece_id=N3UNNLVQJ3GALE77VUERGE4PJDIYJK4T5VJEARML2QSMNJ2AAPJA, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.123209864 (4h52m7.710266105s) [noise=true , address=45.91.116.5:4297 , piece_id=YK7QNB23S43MPEVNDLAT5WBFAMVBPQ3CVYKEMW5VBUHUAI5W4S5Q, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.123571199 (4h52m7.710627439s) [noise=true , address=104.36.82.14:4223 , piece_id=LX33NZFO2TFV43ARCAIGFYXJZZRR633JWZTHSO6XWSRP7TQXU5HQ, seg_pos={0 0}, upload=5025]: Uploading piece... Aug 7 21:18:54.259677990 (4h52m7.846734232s) [upload=5025]: Next returned nil segment Aug 7 21:18:54.259835595 (4h52m7.846891847s) [upload=5025]: Waiting for error group managing segments... Aug 7 21:18:54.269953672 (4h52m7.857009913s) [noise=true , address=45.91.116.7:4011 , piece_id=H2JRFAYC7KLC3TIXQ4PJZTKZLY552JJMZF6XTCFMHFD3RGIUMOCQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.270112598 (4h52m7.857168849s) [noise=true , address=45.91.116.8:4048 , piece_id=2IZI3WOTOBBKYHKVPJDQ6PGS5VTDAFIGTI373VWKT7QLAZR7W3FA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.270947997 (4h52m7.858004239s) [noise=true , address=45.91.116.5:4297 , piece_id=YK7QNB23S43MPEVNDLAT5WBFAMVBPQ3CVYKEMW5VBUHUAI5W4S5Q, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.271046852 (4h52m7.858103094s) [noise=true , address=77.90.31.13:4628 , piece_id=N3UNNLVQJ3GALE77VUERGE4PJDIYJK4T5VJEARML2QSMNJ2AAPJA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.272223499 (4h52m7.85927975s) [noise=true , address=45.91.116.10:4083 , piece_id=HW5YZYWXC7XHQA37JIKXDA7SSO6HOVCCGDS2SHWP7ROPG6MHQ7ZA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.272541713 (4h52m7.859597954s) [noise=true , address=45.91.116.14:4189 , piece_id=GRDSWIHHZ4FEWBEHRY7WPEF546PJWGBIXXCYDESVV4N3GSAWVGYA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.273006260 (4h52m7.860062501s) [noise=true , address=45.91.116.8:4047 , piece_id=7F4MB4X3CJZH3DUXC7VXN3HGQBBRFUV62G4KLCKV7BSARABJENWQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.275575946 (4h52m7.862632188s) [noise=true , address=77.90.31.10:4520 , piece_id=Z5P2H3YFJDC5WMKMFIFVYGFVAU2R27KTNMIAYBDYZGRUT5TWN5QQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.281367938 (4h52m7.868424169s) [noise=true , address=77.90.31.15:4697 , piece_id=PILNZYYWJABT57ITSG77YXQZN7EJKSDHB75RJHE6SPB67FSWCPGQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.284511737 (4h52m7.871567998s) [noise=true , address=77.90.31.19:4841 , piece_id=AUK7XFVC5R73KNHLVVJVRGUTRWBEJ6LP4FZIUJ77DECJRXMHWRRA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.291770877 (4h52m7.878827127s) [noise=true , address=104.36.82.4:4803 , piece_id=DCMHD25G5K5S33FQ72K7VEQ3U6ZV5UBZK5UFVDWUI5I3BX6E4NHA, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.298214294 (4h52m7.885270545s) [noise=true , address=77.90.31.11:4555 , piece_id=D7YHBQ32VTIJEC4HKWQYHWMOULL3KNN7P7TJSA7LV4TLUICYGWBQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.532849875 (4h52m8.119906117s) [noise=true , address=104.36.82.6:4876 , piece_id=6WRCDS47O3FUJLE2BT6QV3OOXKGONXLHZ7TMPXKHH4LP2AJGHV5Q, seg_pos={0 0}, upload=5025]: Done uploading piece. err: <nil> Aug 7 21:18:54.532867749 (4h52m8.11992399s) [seg_pos={0 0}, upload=5025]: Segment reached optimal threshold of 13 pieces. Aug 7 21:18:54.533278306 (4h52m8.120334548s) [noise=true , address=104.36.82.15:4259 , piece_id=ZNJ3QCHTB2LJVM7KI52E3RPGDYEHWWFTEEBBNUIOVPIDOQP7VTKQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Aug 7 21:18:54.533410502 (4h52m8.120466743s) [noise=true , address=104.36.82.15:4258 , piece_id=VT3VKJFUHTSDCNR7PIBIJXEDYG3HGIVT4JZRR5VM3WJ6A5RHQQZQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: expected piece hash; context canceled; EOF Aug 7 21:18:54.533558579 (4h52m8.12061482s) [noise=true , address=104.36.82.14:4223 , piece_id=LX33NZFO2TFV43ARCAIGFYXJZZRR633JWZTHSO6XWSRP7TQXU5HQ, seg_pos={0 0}, upload=5025]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: write tcp 5.161.89.252:47372->104.36.82.14:4223: use of closed network connection; write tcp 5.161.89.252:47372->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47372->104.36.82.14:4223: use of closed network connection Aug 7 21:18:54.533575210 (4h52m8.120631451s) [seg_pos={0 0}, upload=5025]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:18:54.533666891 (4h52m8.120723132s) [upload=5025]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:18:54.533672582 (4h52m8.120728823s) [upload=5025]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:20:54.197308723 (4h54m7.784364965s) [upload=5025]: Done uploading segments... Aug 7 21:20:54.197331416 (4h54m7.784387657s) [upload=5025]: Upload finished. err: metaclient: context deadline exceeded Duration: 2m17.633348968s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...................................................XXXXXXXXX] duration: 30.001160351s FAIL (2025-08-07 21:20:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001181741s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 8 out of 60 is failed (threshold=3) [...................................................XX.XXXXXX] duration: 1m0.001418755s FAIL (2025-08-07 21:20:51 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001480861s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 804.832889ms FAIL (2025-08-07 21:20:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 804.864146ms core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)