Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
389.685785ms
FAIL (2025-08-15 06:28:01 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 15 05:44:22.781048516 (156h18m3.512316206s) [upload=162649]: Starting upload
Aug 15 05:44:22.781143685 (156h18m3.512411345s) [upload=162649]: Done starting upload
Aug 15 05:44:22.781205261 (156h18m3.51247291s) [upload=162649]: Uploading segments...
Aug 15 05:44:22.781552763 (156h18m3.512820423s) [upload=162649]: Got next segment. Inline: false
Aug 15 05:44:22.781613949 (156h18m3.512881639s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 15 05:44:22.781623357 (156h18m3.512891007s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 15 05:44:22.853583217 (156h18m3.584850848s) [seg_pos={0 0}, upload=162649]: Begin upload segment...
Aug 15 05:44:22.853614857 (156h18m3.584882487s) [seg_pos={0 0}, upload=162649]: Done begin upload segment.
Aug 15 05:44:22.853622912 (156h18m3.584890532s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853630877 (156h18m3.584898507s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853960808 (156h18m3.585228438s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854221948 (156h18m3.585489578s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854487648 (156h18m3.585755278s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854706198 (156h18m3.585973818s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854916283 (156h18m3.586183903s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855122541 (156h18m3.586390161s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855323318 (156h18m3.586590939s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855572116 (156h18m3.586839736s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855783854 (156h18m3.587051484s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856043823 (156h18m3.587311442s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856221917 (156h18m3.587489547s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856474260 (156h18m3.587741891s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856684717 (156h18m3.587952347s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856890473 (156h18m3.588158103s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857137107 (156h18m3.588404727s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857407986 (156h18m3.588675616s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.931933679 (156h18m3.663201319s) [upload=162649]: Next returned nil segment
Aug 15 05:44:22.931946533 (156h18m3.663214163s) [upload=162649]: Waiting for error group managing segments...
Aug 15 05:44:22.941125163 (156h18m3.672392803s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.941816452 (156h18m3.673084082s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942102801 (156h18m3.673370441s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942329847 (156h18m3.673597477s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942519253 (156h18m3.673786883s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.944005966 (156h18m3.675273607s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.945343752 (156h18m3.676611392s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.948716731 (156h18m3.679984381s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.954674047 (156h18m3.685941687s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.956262041 (156h18m3.687529671s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.964011115 (156h18m3.695278766s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.973878009 (156h18m3.705145659s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.033905123 (156h18m3.765172773s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039146835 (156h18m3.770414484s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039177702 (156h18m3.770445353s) [seg_pos={0 0}, upload=162649]: Segment reached optimal threshold of 14 pieces.
Aug 15 05:44:23.039726664 (156h18m3.770994314s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHXPzzH8G9xGyABLH7hNnmkZuDdWpoiQkoTxNBzdzNvNQhN29): protocol: read tcp 5.161.89.252:55162->66.23.193.175:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039827213 (156h18m3.771094874s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KNKWRTR6pZfxWFcFwsEKaqGsUMBBSTBsxvmeKsV1GDfEBvQk7): protocol: read tcp 5.161.89.252:18818->79.127.201.215:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039977666 (156h18m3.771245316s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm): protocol: read tcp 5.161.89.252:24186->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.040119473 (156h18m3.771387133s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: context canceled; context canceled
Aug 15 05:44:23.040146273 (156h18m3.771413923s) [seg_pos={0 0}, upload=162649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 15 05:44:23.040241311 (156h18m3.771508962s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 15 05:44:23.040250929 (156h18m3.77151862s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 15 05:44:23.148699216 (156h18m3.879966877s) [upload=162649]: Done uploading segments...
Aug 15 05:44:23.148718513 (156h18m3.879986163s) [upload=162649]: Upload finished. err: <nil>
Duration: 389.705923ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...............................................X......X.....] duration:
228.145841ms
FAIL (2025-08-15 06:27:35 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 15 06:23:13.158332703 (156h56m53.889600322s) [upload=163318]: Starting upload Aug 15 06:23:13.158381314 (156h56m53.889648933s) [upload=163318]: Done starting upload Aug 15 06:23:13.158431398 (156h56m53.889699018s) [upload=163318]: Uploading segments... Aug 15 06:23:13.158487664 (156h56m53.889755294s) [upload=163318]: Got next segment. Inline: true Aug 15 06:23:13.158492433 (156h56m53.889760053s) [upload=163318]: Waiting for error group managing segments... Aug 15 06:23:13.158525365 (156h56m53.889792995s) [upload=163318]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 15 06:23:13.158529061 (156h56m53.889796691s) [upload=163318]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 15 06:23:13.158531456 (156h56m53.889799086s) [upload=163318]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 15 06:23:13.160759987 (156h56m53.892027617s) [upload=163318]: Done uploading segments... Aug 15 06:23:13.160774924 (156h56m53.892042554s) [upload=163318]: Upload finished. err: metaclient: Too Many Requests
Duration: 228.168956ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
338.274ms
FAIL (2025-08-15 06:26:49 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 15 05:44:22.781048516 (156h18m3.512316206s) [upload=162649]: Starting upload
Aug 15 05:44:22.781143685 (156h18m3.512411345s) [upload=162649]: Done starting upload
Aug 15 05:44:22.781205261 (156h18m3.51247291s) [upload=162649]: Uploading segments...
Aug 15 05:44:22.781552763 (156h18m3.512820423s) [upload=162649]: Got next segment. Inline: false
Aug 15 05:44:22.781613949 (156h18m3.512881639s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 15 05:44:22.781623357 (156h18m3.512891007s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 15 05:44:22.853583217 (156h18m3.584850848s) [seg_pos={0 0}, upload=162649]: Begin upload segment...
Aug 15 05:44:22.853614857 (156h18m3.584882487s) [seg_pos={0 0}, upload=162649]: Done begin upload segment.
Aug 15 05:44:22.853622912 (156h18m3.584890532s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853630877 (156h18m3.584898507s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853960808 (156h18m3.585228438s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854221948 (156h18m3.585489578s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854487648 (156h18m3.585755278s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854706198 (156h18m3.585973818s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854916283 (156h18m3.586183903s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855122541 (156h18m3.586390161s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855323318 (156h18m3.586590939s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855572116 (156h18m3.586839736s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855783854 (156h18m3.587051484s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856043823 (156h18m3.587311442s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856221917 (156h18m3.587489547s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856474260 (156h18m3.587741891s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856684717 (156h18m3.587952347s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856890473 (156h18m3.588158103s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857137107 (156h18m3.588404727s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857407986 (156h18m3.588675616s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.931933679 (156h18m3.663201319s) [upload=162649]: Next returned nil segment
Aug 15 05:44:22.931946533 (156h18m3.663214163s) [upload=162649]: Waiting for error group managing segments...
Aug 15 05:44:22.941125163 (156h18m3.672392803s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.941816452 (156h18m3.673084082s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942102801 (156h18m3.673370441s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942329847 (156h18m3.673597477s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942519253 (156h18m3.673786883s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.944005966 (156h18m3.675273607s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.945343752 (156h18m3.676611392s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.948716731 (156h18m3.679984381s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.954674047 (156h18m3.685941687s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.956262041 (156h18m3.687529671s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.964011115 (156h18m3.695278766s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.973878009 (156h18m3.705145659s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.033905123 (156h18m3.765172773s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039146835 (156h18m3.770414484s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039177702 (156h18m3.770445353s) [seg_pos={0 0}, upload=162649]: Segment reached optimal threshold of 14 pieces.
Aug 15 05:44:23.039726664 (156h18m3.770994314s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHXPzzH8G9xGyABLH7hNnmkZuDdWpoiQkoTxNBzdzNvNQhN29): protocol: read tcp 5.161.89.252:55162->66.23.193.175:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039827213 (156h18m3.771094874s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KNKWRTR6pZfxWFcFwsEKaqGsUMBBSTBsxvmeKsV1GDfEBvQk7): protocol: read tcp 5.161.89.252:18818->79.127.201.215:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039977666 (156h18m3.771245316s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm): protocol: read tcp 5.161.89.252:24186->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.040119473 (156h18m3.771387133s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: context canceled; context canceled
Aug 15 05:44:23.040146273 (156h18m3.771413923s) [seg_pos={0 0}, upload=162649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 15 05:44:23.040241311 (156h18m3.771508962s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 15 05:44:23.040250929 (156h18m3.77151862s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 15 05:44:23.148699216 (156h18m3.879966877s) [upload=162649]: Done uploading segments...
Aug 15 05:44:23.148718513 (156h18m3.879986163s) [upload=162649]: Upload finished. err: <nil>
Duration: 338.293929ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [................................................X......X....] duration:
199.595285ms
FAIL (2025-08-15 06:26:30 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 15 06:23:13.158332703 (156h56m53.889600322s) [upload=163318]: Starting upload Aug 15 06:23:13.158381314 (156h56m53.889648933s) [upload=163318]: Done starting upload Aug 15 06:23:13.158431398 (156h56m53.889699018s) [upload=163318]: Uploading segments... Aug 15 06:23:13.158487664 (156h56m53.889755294s) [upload=163318]: Got next segment. Inline: true Aug 15 06:23:13.158492433 (156h56m53.889760053s) [upload=163318]: Waiting for error group managing segments... Aug 15 06:23:13.158525365 (156h56m53.889792995s) [upload=163318]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 15 06:23:13.158529061 (156h56m53.889796691s) [upload=163318]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 15 06:23:13.158531456 (156h56m53.889799086s) [upload=163318]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 15 06:23:13.160759987 (156h56m53.892027617s) [upload=163318]: Done uploading segments... Aug 15 06:23:13.160774924 (156h56m53.892042554s) [upload=163318]: Upload finished. err: metaclient: Too Many Requests
Duration: 199.614682ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
335.584073ms
FAIL (2025-08-15 06:26:12 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 15 05:44:22.781048516 (156h18m3.512316206s) [upload=162649]: Starting upload
Aug 15 05:44:22.781143685 (156h18m3.512411345s) [upload=162649]: Done starting upload
Aug 15 05:44:22.781205261 (156h18m3.51247291s) [upload=162649]: Uploading segments...
Aug 15 05:44:22.781552763 (156h18m3.512820423s) [upload=162649]: Got next segment. Inline: false
Aug 15 05:44:22.781613949 (156h18m3.512881639s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 15 05:44:22.781623357 (156h18m3.512891007s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 15 05:44:22.853583217 (156h18m3.584850848s) [seg_pos={0 0}, upload=162649]: Begin upload segment...
Aug 15 05:44:22.853614857 (156h18m3.584882487s) [seg_pos={0 0}, upload=162649]: Done begin upload segment.
Aug 15 05:44:22.853622912 (156h18m3.584890532s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853630877 (156h18m3.584898507s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853960808 (156h18m3.585228438s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854221948 (156h18m3.585489578s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854487648 (156h18m3.585755278s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854706198 (156h18m3.585973818s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854916283 (156h18m3.586183903s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855122541 (156h18m3.586390161s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855323318 (156h18m3.586590939s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855572116 (156h18m3.586839736s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855783854 (156h18m3.587051484s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856043823 (156h18m3.587311442s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856221917 (156h18m3.587489547s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856474260 (156h18m3.587741891s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856684717 (156h18m3.587952347s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856890473 (156h18m3.588158103s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857137107 (156h18m3.588404727s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857407986 (156h18m3.588675616s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.931933679 (156h18m3.663201319s) [upload=162649]: Next returned nil segment
Aug 15 05:44:22.931946533 (156h18m3.663214163s) [upload=162649]: Waiting for error group managing segments...
Aug 15 05:44:22.941125163 (156h18m3.672392803s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.941816452 (156h18m3.673084082s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942102801 (156h18m3.673370441s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942329847 (156h18m3.673597477s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942519253 (156h18m3.673786883s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.944005966 (156h18m3.675273607s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.945343752 (156h18m3.676611392s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.948716731 (156h18m3.679984381s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.954674047 (156h18m3.685941687s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.956262041 (156h18m3.687529671s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.964011115 (156h18m3.695278766s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.973878009 (156h18m3.705145659s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.033905123 (156h18m3.765172773s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039146835 (156h18m3.770414484s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039177702 (156h18m3.770445353s) [seg_pos={0 0}, upload=162649]: Segment reached optimal threshold of 14 pieces.
Aug 15 05:44:23.039726664 (156h18m3.770994314s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHXPzzH8G9xGyABLH7hNnmkZuDdWpoiQkoTxNBzdzNvNQhN29): protocol: read tcp 5.161.89.252:55162->66.23.193.175:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039827213 (156h18m3.771094874s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KNKWRTR6pZfxWFcFwsEKaqGsUMBBSTBsxvmeKsV1GDfEBvQk7): protocol: read tcp 5.161.89.252:18818->79.127.201.215:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039977666 (156h18m3.771245316s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm): protocol: read tcp 5.161.89.252:24186->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.040119473 (156h18m3.771387133s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: context canceled; context canceled
Aug 15 05:44:23.040146273 (156h18m3.771413923s) [seg_pos={0 0}, upload=162649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 15 05:44:23.040241311 (156h18m3.771508962s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 15 05:44:23.040250929 (156h18m3.77151862s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 15 05:44:23.148699216 (156h18m3.879966877s) [upload=162649]: Done uploading segments...
Aug 15 05:44:23.148718513 (156h18m3.879986163s) [upload=162649]: Upload finished. err: <nil>
Duration: 335.621043ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.................................................X......X...] duration:
298.422509ms
FAIL (2025-08-15 06:25:20 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 15 06:23:13.158332703 (156h56m53.889600322s) [upload=163318]: Starting upload Aug 15 06:23:13.158381314 (156h56m53.889648933s) [upload=163318]: Done starting upload Aug 15 06:23:13.158431398 (156h56m53.889699018s) [upload=163318]: Uploading segments... Aug 15 06:23:13.158487664 (156h56m53.889755294s) [upload=163318]: Got next segment. Inline: true Aug 15 06:23:13.158492433 (156h56m53.889760053s) [upload=163318]: Waiting for error group managing segments... Aug 15 06:23:13.158525365 (156h56m53.889792995s) [upload=163318]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 15 06:23:13.158529061 (156h56m53.889796691s) [upload=163318]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 15 06:23:13.158531456 (156h56m53.889799086s) [upload=163318]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 15 06:23:13.160759987 (156h56m53.892027617s) [upload=163318]: Done uploading segments... Aug 15 06:23:13.160774924 (156h56m53.892042554s) [upload=163318]: Upload finished. err: metaclient: Too Many Requests
Duration: 298.444451ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
412.722312ms
FAIL (2025-08-15 06:25:00 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 15 05:44:22.781048516 (156h18m3.512316206s) [upload=162649]: Starting upload
Aug 15 05:44:22.781143685 (156h18m3.512411345s) [upload=162649]: Done starting upload
Aug 15 05:44:22.781205261 (156h18m3.51247291s) [upload=162649]: Uploading segments...
Aug 15 05:44:22.781552763 (156h18m3.512820423s) [upload=162649]: Got next segment. Inline: false
Aug 15 05:44:22.781613949 (156h18m3.512881639s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 15 05:44:22.781623357 (156h18m3.512891007s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 15 05:44:22.853583217 (156h18m3.584850848s) [seg_pos={0 0}, upload=162649]: Begin upload segment...
Aug 15 05:44:22.853614857 (156h18m3.584882487s) [seg_pos={0 0}, upload=162649]: Done begin upload segment.
Aug 15 05:44:22.853622912 (156h18m3.584890532s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853630877 (156h18m3.584898507s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853960808 (156h18m3.585228438s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854221948 (156h18m3.585489578s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854487648 (156h18m3.585755278s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854706198 (156h18m3.585973818s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854916283 (156h18m3.586183903s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855122541 (156h18m3.586390161s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855323318 (156h18m3.586590939s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855572116 (156h18m3.586839736s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855783854 (156h18m3.587051484s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856043823 (156h18m3.587311442s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856221917 (156h18m3.587489547s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856474260 (156h18m3.587741891s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856684717 (156h18m3.587952347s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856890473 (156h18m3.588158103s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857137107 (156h18m3.588404727s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857407986 (156h18m3.588675616s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.931933679 (156h18m3.663201319s) [upload=162649]: Next returned nil segment
Aug 15 05:44:22.931946533 (156h18m3.663214163s) [upload=162649]: Waiting for error group managing segments...
Aug 15 05:44:22.941125163 (156h18m3.672392803s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.941816452 (156h18m3.673084082s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942102801 (156h18m3.673370441s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942329847 (156h18m3.673597477s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942519253 (156h18m3.673786883s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.944005966 (156h18m3.675273607s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.945343752 (156h18m3.676611392s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.948716731 (156h18m3.679984381s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.954674047 (156h18m3.685941687s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.956262041 (156h18m3.687529671s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.964011115 (156h18m3.695278766s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.973878009 (156h18m3.705145659s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.033905123 (156h18m3.765172773s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039146835 (156h18m3.770414484s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039177702 (156h18m3.770445353s) [seg_pos={0 0}, upload=162649]: Segment reached optimal threshold of 14 pieces.
Aug 15 05:44:23.039726664 (156h18m3.770994314s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHXPzzH8G9xGyABLH7hNnmkZuDdWpoiQkoTxNBzdzNvNQhN29): protocol: read tcp 5.161.89.252:55162->66.23.193.175:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039827213 (156h18m3.771094874s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KNKWRTR6pZfxWFcFwsEKaqGsUMBBSTBsxvmeKsV1GDfEBvQk7): protocol: read tcp 5.161.89.252:18818->79.127.201.215:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039977666 (156h18m3.771245316s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm): protocol: read tcp 5.161.89.252:24186->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.040119473 (156h18m3.771387133s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: context canceled; context canceled
Aug 15 05:44:23.040146273 (156h18m3.771413923s) [seg_pos={0 0}, upload=162649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 15 05:44:23.040241311 (156h18m3.771508962s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 15 05:44:23.040250929 (156h18m3.77151862s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 15 05:44:23.148699216 (156h18m3.879966877s) [upload=162649]: Done uploading segments...
Aug 15 05:44:23.148718513 (156h18m3.879986163s) [upload=162649]: Upload finished. err: <nil>
Duration: 412.748881ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..................................................X......X..] duration:
144.605208ms
FAIL (2025-08-15 06:24:41 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 15 06:23:13.158332703 (156h56m53.889600322s) [upload=163318]: Starting upload Aug 15 06:23:13.158381314 (156h56m53.889648933s) [upload=163318]: Done starting upload Aug 15 06:23:13.158431398 (156h56m53.889699018s) [upload=163318]: Uploading segments... Aug 15 06:23:13.158487664 (156h56m53.889755294s) [upload=163318]: Got next segment. Inline: true Aug 15 06:23:13.158492433 (156h56m53.889760053s) [upload=163318]: Waiting for error group managing segments... Aug 15 06:23:13.158525365 (156h56m53.889792995s) [upload=163318]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 15 06:23:13.158529061 (156h56m53.889796691s) [upload=163318]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 15 06:23:13.158531456 (156h56m53.889799086s) [upload=163318]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 15 06:23:13.160759987 (156h56m53.892027617s) [upload=163318]: Done uploading segments... Aug 15 06:23:13.160774924 (156h56m53.892042554s) [upload=163318]: Upload finished. err: metaclient: Too Many Requests
Duration: 144.625897ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...................................................X......X.] duration:
140.058585ms
FAIL (2025-08-15 06:24:04 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 15 06:23:13.158332703 (156h56m53.889600322s) [upload=163318]: Starting upload Aug 15 06:23:13.158381314 (156h56m53.889648933s) [upload=163318]: Done starting upload Aug 15 06:23:13.158431398 (156h56m53.889699018s) [upload=163318]: Uploading segments... Aug 15 06:23:13.158487664 (156h56m53.889755294s) [upload=163318]: Got next segment. Inline: true Aug 15 06:23:13.158492433 (156h56m53.889760053s) [upload=163318]: Waiting for error group managing segments... Aug 15 06:23:13.158525365 (156h56m53.889792995s) [upload=163318]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 15 06:23:13.158529061 (156h56m53.889796691s) [upload=163318]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 15 06:23:13.158531456 (156h56m53.889799086s) [upload=163318]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 15 06:23:13.160759987 (156h56m53.892027617s) [upload=163318]: Done uploading segments... Aug 15 06:23:13.160774924 (156h56m53.892042554s) [upload=163318]: Upload finished. err: metaclient: Too Many Requests
Duration: 140.078032ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
324.530157ms
FAIL (2025-08-15 06:23:58 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 15 05:44:22.781048516 (156h18m3.512316206s) [upload=162649]: Starting upload
Aug 15 05:44:22.781143685 (156h18m3.512411345s) [upload=162649]: Done starting upload
Aug 15 05:44:22.781205261 (156h18m3.51247291s) [upload=162649]: Uploading segments...
Aug 15 05:44:22.781552763 (156h18m3.512820423s) [upload=162649]: Got next segment. Inline: false
Aug 15 05:44:22.781613949 (156h18m3.512881639s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 15 05:44:22.781623357 (156h18m3.512891007s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 15 05:44:22.853583217 (156h18m3.584850848s) [seg_pos={0 0}, upload=162649]: Begin upload segment...
Aug 15 05:44:22.853614857 (156h18m3.584882487s) [seg_pos={0 0}, upload=162649]: Done begin upload segment.
Aug 15 05:44:22.853622912 (156h18m3.584890532s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853630877 (156h18m3.584898507s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853960808 (156h18m3.585228438s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854221948 (156h18m3.585489578s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854487648 (156h18m3.585755278s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854706198 (156h18m3.585973818s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854916283 (156h18m3.586183903s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855122541 (156h18m3.586390161s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855323318 (156h18m3.586590939s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855572116 (156h18m3.586839736s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855783854 (156h18m3.587051484s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856043823 (156h18m3.587311442s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856221917 (156h18m3.587489547s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856474260 (156h18m3.587741891s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856684717 (156h18m3.587952347s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856890473 (156h18m3.588158103s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857137107 (156h18m3.588404727s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857407986 (156h18m3.588675616s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.931933679 (156h18m3.663201319s) [upload=162649]: Next returned nil segment
Aug 15 05:44:22.931946533 (156h18m3.663214163s) [upload=162649]: Waiting for error group managing segments...
Aug 15 05:44:22.941125163 (156h18m3.672392803s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.941816452 (156h18m3.673084082s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942102801 (156h18m3.673370441s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942329847 (156h18m3.673597477s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942519253 (156h18m3.673786883s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.944005966 (156h18m3.675273607s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.945343752 (156h18m3.676611392s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.948716731 (156h18m3.679984381s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.954674047 (156h18m3.685941687s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.956262041 (156h18m3.687529671s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.964011115 (156h18m3.695278766s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.973878009 (156h18m3.705145659s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.033905123 (156h18m3.765172773s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039146835 (156h18m3.770414484s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039177702 (156h18m3.770445353s) [seg_pos={0 0}, upload=162649]: Segment reached optimal threshold of 14 pieces.
Aug 15 05:44:23.039726664 (156h18m3.770994314s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHXPzzH8G9xGyABLH7hNnmkZuDdWpoiQkoTxNBzdzNvNQhN29): protocol: read tcp 5.161.89.252:55162->66.23.193.175:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039827213 (156h18m3.771094874s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KNKWRTR6pZfxWFcFwsEKaqGsUMBBSTBsxvmeKsV1GDfEBvQk7): protocol: read tcp 5.161.89.252:18818->79.127.201.215:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039977666 (156h18m3.771245316s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm): protocol: read tcp 5.161.89.252:24186->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.040119473 (156h18m3.771387133s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: context canceled; context canceled
Aug 15 05:44:23.040146273 (156h18m3.771413923s) [seg_pos={0 0}, upload=162649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 15 05:44:23.040241311 (156h18m3.771508962s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 15 05:44:23.040250929 (156h18m3.77151862s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 15 05:44:23.148699216 (156h18m3.879966877s) [upload=162649]: Done uploading segments...
Aug 15 05:44:23.148718513 (156h18m3.879986163s) [upload=162649]: Upload finished. err: <nil>
Duration: 324.56306ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [....................................................X......X] duration:
13.81147ms
FAIL (2025-08-15 06:23:13 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 15 06:23:13.158332703 (156h56m53.889600322s) [upload=163318]: Starting upload Aug 15 06:23:13.158381314 (156h56m53.889648933s) [upload=163318]: Done starting upload Aug 15 06:23:13.158431398 (156h56m53.889699018s) [upload=163318]: Uploading segments... Aug 15 06:23:13.158487664 (156h56m53.889755294s) [upload=163318]: Got next segment. Inline: true Aug 15 06:23:13.158492433 (156h56m53.889760053s) [upload=163318]: Waiting for error group managing segments... Aug 15 06:23:13.158525365 (156h56m53.889792995s) [upload=163318]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 15 06:23:13.158529061 (156h56m53.889796691s) [upload=163318]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 15 06:23:13.158531456 (156h56m53.889799086s) [upload=163318]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 15 06:23:13.160759987 (156h56m53.892027617s) [upload=163318]: Done uploading segments... Aug 15 06:23:13.160774924 (156h56m53.892042554s) [upload=163318]: Upload finished. err: metaclient: Too Many Requests
Duration: 13.829424ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
346.415262ms
FAIL (2025-08-15 06:23:03 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 15 05:44:22.781048516 (156h18m3.512316206s) [upload=162649]: Starting upload
Aug 15 05:44:22.781143685 (156h18m3.512411345s) [upload=162649]: Done starting upload
Aug 15 05:44:22.781205261 (156h18m3.51247291s) [upload=162649]: Uploading segments...
Aug 15 05:44:22.781552763 (156h18m3.512820423s) [upload=162649]: Got next segment. Inline: false
Aug 15 05:44:22.781613949 (156h18m3.512881639s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 15 05:44:22.781623357 (156h18m3.512891007s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 15 05:44:22.853583217 (156h18m3.584850848s) [seg_pos={0 0}, upload=162649]: Begin upload segment...
Aug 15 05:44:22.853614857 (156h18m3.584882487s) [seg_pos={0 0}, upload=162649]: Done begin upload segment.
Aug 15 05:44:22.853622912 (156h18m3.584890532s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853630877 (156h18m3.584898507s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853960808 (156h18m3.585228438s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854221948 (156h18m3.585489578s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854487648 (156h18m3.585755278s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854706198 (156h18m3.585973818s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854916283 (156h18m3.586183903s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855122541 (156h18m3.586390161s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855323318 (156h18m3.586590939s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855572116 (156h18m3.586839736s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855783854 (156h18m3.587051484s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856043823 (156h18m3.587311442s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856221917 (156h18m3.587489547s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856474260 (156h18m3.587741891s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856684717 (156h18m3.587952347s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856890473 (156h18m3.588158103s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857137107 (156h18m3.588404727s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857407986 (156h18m3.588675616s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.931933679 (156h18m3.663201319s) [upload=162649]: Next returned nil segment
Aug 15 05:44:22.931946533 (156h18m3.663214163s) [upload=162649]: Waiting for error group managing segments...
Aug 15 05:44:22.941125163 (156h18m3.672392803s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.941816452 (156h18m3.673084082s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942102801 (156h18m3.673370441s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942329847 (156h18m3.673597477s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942519253 (156h18m3.673786883s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.944005966 (156h18m3.675273607s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.945343752 (156h18m3.676611392s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.948716731 (156h18m3.679984381s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.954674047 (156h18m3.685941687s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.956262041 (156h18m3.687529671s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.964011115 (156h18m3.695278766s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.973878009 (156h18m3.705145659s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.033905123 (156h18m3.765172773s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039146835 (156h18m3.770414484s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039177702 (156h18m3.770445353s) [seg_pos={0 0}, upload=162649]: Segment reached optimal threshold of 14 pieces.
Aug 15 05:44:23.039726664 (156h18m3.770994314s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHXPzzH8G9xGyABLH7hNnmkZuDdWpoiQkoTxNBzdzNvNQhN29): protocol: read tcp 5.161.89.252:55162->66.23.193.175:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039827213 (156h18m3.771094874s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KNKWRTR6pZfxWFcFwsEKaqGsUMBBSTBsxvmeKsV1GDfEBvQk7): protocol: read tcp 5.161.89.252:18818->79.127.201.215:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039977666 (156h18m3.771245316s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm): protocol: read tcp 5.161.89.252:24186->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.040119473 (156h18m3.771387133s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: context canceled; context canceled
Aug 15 05:44:23.040146273 (156h18m3.771413923s) [seg_pos={0 0}, upload=162649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 15 05:44:23.040241311 (156h18m3.771508962s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 15 05:44:23.040250929 (156h18m3.77151862s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 15 05:44:23.148699216 (156h18m3.879966877s) [upload=162649]: Done uploading segments...
Aug 15 05:44:23.148718513 (156h18m3.879986163s) [upload=162649]: Upload finished. err: <nil>
Duration: 346.440469ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
506.814696ms
FAIL (2025-08-15 06:22:32 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 15 05:44:22.781048516 (156h18m3.512316206s) [upload=162649]: Starting upload
Aug 15 05:44:22.781143685 (156h18m3.512411345s) [upload=162649]: Done starting upload
Aug 15 05:44:22.781205261 (156h18m3.51247291s) [upload=162649]: Uploading segments...
Aug 15 05:44:22.781552763 (156h18m3.512820423s) [upload=162649]: Got next segment. Inline: false
Aug 15 05:44:22.781613949 (156h18m3.512881639s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 15 05:44:22.781623357 (156h18m3.512891007s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 15 05:44:22.853583217 (156h18m3.584850848s) [seg_pos={0 0}, upload=162649]: Begin upload segment...
Aug 15 05:44:22.853614857 (156h18m3.584882487s) [seg_pos={0 0}, upload=162649]: Done begin upload segment.
Aug 15 05:44:22.853622912 (156h18m3.584890532s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853630877 (156h18m3.584898507s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853960808 (156h18m3.585228438s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854221948 (156h18m3.585489578s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854487648 (156h18m3.585755278s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854706198 (156h18m3.585973818s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854916283 (156h18m3.586183903s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855122541 (156h18m3.586390161s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855323318 (156h18m3.586590939s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855572116 (156h18m3.586839736s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855783854 (156h18m3.587051484s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856043823 (156h18m3.587311442s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856221917 (156h18m3.587489547s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856474260 (156h18m3.587741891s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856684717 (156h18m3.587952347s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856890473 (156h18m3.588158103s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857137107 (156h18m3.588404727s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857407986 (156h18m3.588675616s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.931933679 (156h18m3.663201319s) [upload=162649]: Next returned nil segment
Aug 15 05:44:22.931946533 (156h18m3.663214163s) [upload=162649]: Waiting for error group managing segments...
Aug 15 05:44:22.941125163 (156h18m3.672392803s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.941816452 (156h18m3.673084082s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942102801 (156h18m3.673370441s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942329847 (156h18m3.673597477s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942519253 (156h18m3.673786883s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.944005966 (156h18m3.675273607s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.945343752 (156h18m3.676611392s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.948716731 (156h18m3.679984381s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.954674047 (156h18m3.685941687s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.956262041 (156h18m3.687529671s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.964011115 (156h18m3.695278766s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.973878009 (156h18m3.705145659s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.033905123 (156h18m3.765172773s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039146835 (156h18m3.770414484s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039177702 (156h18m3.770445353s) [seg_pos={0 0}, upload=162649]: Segment reached optimal threshold of 14 pieces.
Aug 15 05:44:23.039726664 (156h18m3.770994314s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHXPzzH8G9xGyABLH7hNnmkZuDdWpoiQkoTxNBzdzNvNQhN29): protocol: read tcp 5.161.89.252:55162->66.23.193.175:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039827213 (156h18m3.771094874s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KNKWRTR6pZfxWFcFwsEKaqGsUMBBSTBsxvmeKsV1GDfEBvQk7): protocol: read tcp 5.161.89.252:18818->79.127.201.215:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039977666 (156h18m3.771245316s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm): protocol: read tcp 5.161.89.252:24186->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.040119473 (156h18m3.771387133s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: context canceled; context canceled
Aug 15 05:44:23.040146273 (156h18m3.771413923s) [seg_pos={0 0}, upload=162649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 15 05:44:23.040241311 (156h18m3.771508962s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 15 05:44:23.040250929 (156h18m3.77151862s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 15 05:44:23.148699216 (156h18m3.879966877s) [upload=162649]: Done uploading segments...
Aug 15 05:44:23.148718513 (156h18m3.879986163s) [upload=162649]: Upload finished. err: <nil>
Duration: 506.834263ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
120.521613ms
FAIL (2025-08-15 06:22:12 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 15 06:17:46.487011506 (156h51m27.218279167s) [upload=163229]: Starting upload Aug 15 06:17:46.487080387 (156h51m27.218348037s) [upload=163229]: Done starting upload Aug 15 06:17:46.487149736 (156h51m27.218417387s) [upload=163229]: Uploading segments... Aug 15 06:17:46.487218095 (156h51m27.218485755s) [upload=163229]: Got next segment. Inline: true Aug 15 06:17:46.487225218 (156h51m27.218492868s) [upload=163229]: Waiting for error group managing segments... Aug 15 06:17:46.487270695 (156h51m27.218538345s) [upload=163229]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 15 06:17:46.487275964 (156h51m27.218543615s) [upload=163229]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 15 06:17:46.487279381 (156h51m27.218547031s) [upload=163229]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 15 06:17:46.490074687 (156h51m27.221342337s) [upload=163229]: Done uploading segments... Aug 15 06:17:46.490086849 (156h51m27.2213545s) [upload=163229]: Upload finished. err: metaclient: Too Many Requests
Duration: 120.560526ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
149.007033ms
FAIL (2025-08-15 06:21:46 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 15 06:17:46.487011506 (156h51m27.218279167s) [upload=163229]: Starting upload Aug 15 06:17:46.487080387 (156h51m27.218348037s) [upload=163229]: Done starting upload Aug 15 06:17:46.487149736 (156h51m27.218417387s) [upload=163229]: Uploading segments... Aug 15 06:17:46.487218095 (156h51m27.218485755s) [upload=163229]: Got next segment. Inline: true Aug 15 06:17:46.487225218 (156h51m27.218492868s) [upload=163229]: Waiting for error group managing segments... Aug 15 06:17:46.487270695 (156h51m27.218538345s) [upload=163229]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 15 06:17:46.487275964 (156h51m27.218543615s) [upload=163229]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 15 06:17:46.487279381 (156h51m27.218547031s) [upload=163229]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 15 06:17:46.490074687 (156h51m27.221342337s) [upload=163229]: Done uploading segments... Aug 15 06:17:46.490086849 (156h51m27.2213545s) [upload=163229]: Upload finished. err: metaclient: Too Many Requests
Duration: 149.028344ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
454.555441ms
FAIL (2025-08-15 06:21:23 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 15 05:44:22.781048516 (156h18m3.512316206s) [upload=162649]: Starting upload
Aug 15 05:44:22.781143685 (156h18m3.512411345s) [upload=162649]: Done starting upload
Aug 15 05:44:22.781205261 (156h18m3.51247291s) [upload=162649]: Uploading segments...
Aug 15 05:44:22.781552763 (156h18m3.512820423s) [upload=162649]: Got next segment. Inline: false
Aug 15 05:44:22.781613949 (156h18m3.512881639s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 15 05:44:22.781623357 (156h18m3.512891007s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 15 05:44:22.853583217 (156h18m3.584850848s) [seg_pos={0 0}, upload=162649]: Begin upload segment...
Aug 15 05:44:22.853614857 (156h18m3.584882487s) [seg_pos={0 0}, upload=162649]: Done begin upload segment.
Aug 15 05:44:22.853622912 (156h18m3.584890532s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853630877 (156h18m3.584898507s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853960808 (156h18m3.585228438s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854221948 (156h18m3.585489578s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854487648 (156h18m3.585755278s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854706198 (156h18m3.585973818s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854916283 (156h18m3.586183903s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855122541 (156h18m3.586390161s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855323318 (156h18m3.586590939s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855572116 (156h18m3.586839736s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855783854 (156h18m3.587051484s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856043823 (156h18m3.587311442s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856221917 (156h18m3.587489547s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856474260 (156h18m3.587741891s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856684717 (156h18m3.587952347s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856890473 (156h18m3.588158103s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857137107 (156h18m3.588404727s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857407986 (156h18m3.588675616s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.931933679 (156h18m3.663201319s) [upload=162649]: Next returned nil segment
Aug 15 05:44:22.931946533 (156h18m3.663214163s) [upload=162649]: Waiting for error group managing segments...
Aug 15 05:44:22.941125163 (156h18m3.672392803s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.941816452 (156h18m3.673084082s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942102801 (156h18m3.673370441s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942329847 (156h18m3.673597477s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942519253 (156h18m3.673786883s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.944005966 (156h18m3.675273607s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.945343752 (156h18m3.676611392s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.948716731 (156h18m3.679984381s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.954674047 (156h18m3.685941687s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.956262041 (156h18m3.687529671s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.964011115 (156h18m3.695278766s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.973878009 (156h18m3.705145659s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.033905123 (156h18m3.765172773s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039146835 (156h18m3.770414484s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039177702 (156h18m3.770445353s) [seg_pos={0 0}, upload=162649]: Segment reached optimal threshold of 14 pieces.
Aug 15 05:44:23.039726664 (156h18m3.770994314s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHXPzzH8G9xGyABLH7hNnmkZuDdWpoiQkoTxNBzdzNvNQhN29): protocol: read tcp 5.161.89.252:55162->66.23.193.175:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039827213 (156h18m3.771094874s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KNKWRTR6pZfxWFcFwsEKaqGsUMBBSTBsxvmeKsV1GDfEBvQk7): protocol: read tcp 5.161.89.252:18818->79.127.201.215:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039977666 (156h18m3.771245316s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm): protocol: read tcp 5.161.89.252:24186->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.040119473 (156h18m3.771387133s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: context canceled; context canceled
Aug 15 05:44:23.040146273 (156h18m3.771413923s) [seg_pos={0 0}, upload=162649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 15 05:44:23.040241311 (156h18m3.771508962s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 15 05:44:23.040250929 (156h18m3.77151862s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 15 05:44:23.148699216 (156h18m3.879966877s) [upload=162649]: Done uploading segments...
Aug 15 05:44:23.148718513 (156h18m3.879986163s) [upload=162649]: Upload finished. err: <nil>
Duration: 454.577572ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [X......................................................X....] duration:
120.90925ms
FAIL (2025-08-15 06:21:04 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 15 06:17:46.487011506 (156h51m27.218279167s) [upload=163229]: Starting upload Aug 15 06:17:46.487080387 (156h51m27.218348037s) [upload=163229]: Done starting upload Aug 15 06:17:46.487149736 (156h51m27.218417387s) [upload=163229]: Uploading segments... Aug 15 06:17:46.487218095 (156h51m27.218485755s) [upload=163229]: Got next segment. Inline: true Aug 15 06:17:46.487225218 (156h51m27.218492868s) [upload=163229]: Waiting for error group managing segments... Aug 15 06:17:46.487270695 (156h51m27.218538345s) [upload=163229]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 15 06:17:46.487275964 (156h51m27.218543615s) [upload=163229]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 15 06:17:46.487279381 (156h51m27.218547031s) [upload=163229]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 15 06:17:46.490074687 (156h51m27.221342337s) [upload=163229]: Done uploading segments... Aug 15 06:17:46.490086849 (156h51m27.2213545s) [upload=163229]: Upload finished. err: metaclient: Too Many Requests
Duration: 120.929018ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.X......................................................X...] duration:
394.267333ms
FAIL (2025-08-15 06:20:30 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 15 06:17:46.487011506 (156h51m27.218279167s) [upload=163229]: Starting upload Aug 15 06:17:46.487080387 (156h51m27.218348037s) [upload=163229]: Done starting upload Aug 15 06:17:46.487149736 (156h51m27.218417387s) [upload=163229]: Uploading segments... Aug 15 06:17:46.487218095 (156h51m27.218485755s) [upload=163229]: Got next segment. Inline: true Aug 15 06:17:46.487225218 (156h51m27.218492868s) [upload=163229]: Waiting for error group managing segments... Aug 15 06:17:46.487270695 (156h51m27.218538345s) [upload=163229]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 15 06:17:46.487275964 (156h51m27.218543615s) [upload=163229]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 15 06:17:46.487279381 (156h51m27.218547031s) [upload=163229]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 15 06:17:46.490074687 (156h51m27.221342337s) [upload=163229]: Done uploading segments... Aug 15 06:17:46.490086849 (156h51m27.2213545s) [upload=163229]: Upload finished. err: metaclient: Too Many Requests
Duration: 394.292519ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
585.569142ms
FAIL (2025-08-15 06:20:25 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 15 05:44:22.781048516 (156h18m3.512316206s) [upload=162649]: Starting upload
Aug 15 05:44:22.781143685 (156h18m3.512411345s) [upload=162649]: Done starting upload
Aug 15 05:44:22.781205261 (156h18m3.51247291s) [upload=162649]: Uploading segments...
Aug 15 05:44:22.781552763 (156h18m3.512820423s) [upload=162649]: Got next segment. Inline: false
Aug 15 05:44:22.781613949 (156h18m3.512881639s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 15 05:44:22.781623357 (156h18m3.512891007s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 15 05:44:22.853583217 (156h18m3.584850848s) [seg_pos={0 0}, upload=162649]: Begin upload segment...
Aug 15 05:44:22.853614857 (156h18m3.584882487s) [seg_pos={0 0}, upload=162649]: Done begin upload segment.
Aug 15 05:44:22.853622912 (156h18m3.584890532s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853630877 (156h18m3.584898507s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853960808 (156h18m3.585228438s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854221948 (156h18m3.585489578s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854487648 (156h18m3.585755278s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854706198 (156h18m3.585973818s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854916283 (156h18m3.586183903s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855122541 (156h18m3.586390161s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855323318 (156h18m3.586590939s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855572116 (156h18m3.586839736s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855783854 (156h18m3.587051484s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856043823 (156h18m3.587311442s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856221917 (156h18m3.587489547s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856474260 (156h18m3.587741891s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856684717 (156h18m3.587952347s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856890473 (156h18m3.588158103s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857137107 (156h18m3.588404727s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857407986 (156h18m3.588675616s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.931933679 (156h18m3.663201319s) [upload=162649]: Next returned nil segment
Aug 15 05:44:22.931946533 (156h18m3.663214163s) [upload=162649]: Waiting for error group managing segments...
Aug 15 05:44:22.941125163 (156h18m3.672392803s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.941816452 (156h18m3.673084082s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942102801 (156h18m3.673370441s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942329847 (156h18m3.673597477s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942519253 (156h18m3.673786883s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.944005966 (156h18m3.675273607s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.945343752 (156h18m3.676611392s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.948716731 (156h18m3.679984381s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.954674047 (156h18m3.685941687s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.956262041 (156h18m3.687529671s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.964011115 (156h18m3.695278766s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.973878009 (156h18m3.705145659s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.033905123 (156h18m3.765172773s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039146835 (156h18m3.770414484s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039177702 (156h18m3.770445353s) [seg_pos={0 0}, upload=162649]: Segment reached optimal threshold of 14 pieces.
Aug 15 05:44:23.039726664 (156h18m3.770994314s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHXPzzH8G9xGyABLH7hNnmkZuDdWpoiQkoTxNBzdzNvNQhN29): protocol: read tcp 5.161.89.252:55162->66.23.193.175:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039827213 (156h18m3.771094874s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KNKWRTR6pZfxWFcFwsEKaqGsUMBBSTBsxvmeKsV1GDfEBvQk7): protocol: read tcp 5.161.89.252:18818->79.127.201.215:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039977666 (156h18m3.771245316s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm): protocol: read tcp 5.161.89.252:24186->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.040119473 (156h18m3.771387133s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: context canceled; context canceled
Aug 15 05:44:23.040146273 (156h18m3.771413923s) [seg_pos={0 0}, upload=162649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 15 05:44:23.040241311 (156h18m3.771508962s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 15 05:44:23.040250929 (156h18m3.77151862s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 15 05:44:23.148699216 (156h18m3.879966877s) [upload=162649]: Done uploading segments...
Aug 15 05:44:23.148718513 (156h18m3.879986163s) [upload=162649]: Upload finished. err: <nil>
Duration: 585.585492ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
320.4297ms
FAIL (2025-08-15 06:19:52 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 15 05:44:22.781048516 (156h18m3.512316206s) [upload=162649]: Starting upload
Aug 15 05:44:22.781143685 (156h18m3.512411345s) [upload=162649]: Done starting upload
Aug 15 05:44:22.781205261 (156h18m3.51247291s) [upload=162649]: Uploading segments...
Aug 15 05:44:22.781552763 (156h18m3.512820423s) [upload=162649]: Got next segment. Inline: false
Aug 15 05:44:22.781613949 (156h18m3.512881639s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 15 05:44:22.781623357 (156h18m3.512891007s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 15 05:44:22.853583217 (156h18m3.584850848s) [seg_pos={0 0}, upload=162649]: Begin upload segment...
Aug 15 05:44:22.853614857 (156h18m3.584882487s) [seg_pos={0 0}, upload=162649]: Done begin upload segment.
Aug 15 05:44:22.853622912 (156h18m3.584890532s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853630877 (156h18m3.584898507s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853960808 (156h18m3.585228438s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854221948 (156h18m3.585489578s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854487648 (156h18m3.585755278s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854706198 (156h18m3.585973818s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854916283 (156h18m3.586183903s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855122541 (156h18m3.586390161s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855323318 (156h18m3.586590939s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855572116 (156h18m3.586839736s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855783854 (156h18m3.587051484s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856043823 (156h18m3.587311442s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856221917 (156h18m3.587489547s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856474260 (156h18m3.587741891s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856684717 (156h18m3.587952347s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856890473 (156h18m3.588158103s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857137107 (156h18m3.588404727s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857407986 (156h18m3.588675616s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.931933679 (156h18m3.663201319s) [upload=162649]: Next returned nil segment
Aug 15 05:44:22.931946533 (156h18m3.663214163s) [upload=162649]: Waiting for error group managing segments...
Aug 15 05:44:22.941125163 (156h18m3.672392803s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.941816452 (156h18m3.673084082s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942102801 (156h18m3.673370441s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942329847 (156h18m3.673597477s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942519253 (156h18m3.673786883s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.944005966 (156h18m3.675273607s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.945343752 (156h18m3.676611392s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.948716731 (156h18m3.679984381s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.954674047 (156h18m3.685941687s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.956262041 (156h18m3.687529671s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.964011115 (156h18m3.695278766s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.973878009 (156h18m3.705145659s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.033905123 (156h18m3.765172773s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039146835 (156h18m3.770414484s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039177702 (156h18m3.770445353s) [seg_pos={0 0}, upload=162649]: Segment reached optimal threshold of 14 pieces.
Aug 15 05:44:23.039726664 (156h18m3.770994314s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHXPzzH8G9xGyABLH7hNnmkZuDdWpoiQkoTxNBzdzNvNQhN29): protocol: read tcp 5.161.89.252:55162->66.23.193.175:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039827213 (156h18m3.771094874s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KNKWRTR6pZfxWFcFwsEKaqGsUMBBSTBsxvmeKsV1GDfEBvQk7): protocol: read tcp 5.161.89.252:18818->79.127.201.215:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039977666 (156h18m3.771245316s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm): protocol: read tcp 5.161.89.252:24186->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.040119473 (156h18m3.771387133s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: context canceled; context canceled
Aug 15 05:44:23.040146273 (156h18m3.771413923s) [seg_pos={0 0}, upload=162649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 15 05:44:23.040241311 (156h18m3.771508962s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 15 05:44:23.040250929 (156h18m3.77151862s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 15 05:44:23.148699216 (156h18m3.879966877s) [upload=162649]: Done uploading segments...
Aug 15 05:44:23.148718513 (156h18m3.879986163s) [upload=162649]: Upload finished. err: <nil>
Duration: 320.45125ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..X......................................................X..] duration:
132.116344ms
FAIL (2025-08-15 06:19:19 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 15 06:17:46.487011506 (156h51m27.218279167s) [upload=163229]: Starting upload Aug 15 06:17:46.487080387 (156h51m27.218348037s) [upload=163229]: Done starting upload Aug 15 06:17:46.487149736 (156h51m27.218417387s) [upload=163229]: Uploading segments... Aug 15 06:17:46.487218095 (156h51m27.218485755s) [upload=163229]: Got next segment. Inline: true Aug 15 06:17:46.487225218 (156h51m27.218492868s) [upload=163229]: Waiting for error group managing segments... Aug 15 06:17:46.487270695 (156h51m27.218538345s) [upload=163229]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 15 06:17:46.487275964 (156h51m27.218543615s) [upload=163229]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 15 06:17:46.487279381 (156h51m27.218547031s) [upload=163229]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 15 06:17:46.490074687 (156h51m27.221342337s) [upload=163229]: Done uploading segments... Aug 15 06:17:46.490086849 (156h51m27.2213545s) [upload=163229]: Upload finished. err: metaclient: Too Many Requests
Duration: 132.140409ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
401.427963ms
FAIL (2025-08-15 06:19:14 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 15 05:44:22.781048516 (156h18m3.512316206s) [upload=162649]: Starting upload
Aug 15 05:44:22.781143685 (156h18m3.512411345s) [upload=162649]: Done starting upload
Aug 15 05:44:22.781205261 (156h18m3.51247291s) [upload=162649]: Uploading segments...
Aug 15 05:44:22.781552763 (156h18m3.512820423s) [upload=162649]: Got next segment. Inline: false
Aug 15 05:44:22.781613949 (156h18m3.512881639s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 15 05:44:22.781623357 (156h18m3.512891007s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 15 05:44:22.853583217 (156h18m3.584850848s) [seg_pos={0 0}, upload=162649]: Begin upload segment...
Aug 15 05:44:22.853614857 (156h18m3.584882487s) [seg_pos={0 0}, upload=162649]: Done begin upload segment.
Aug 15 05:44:22.853622912 (156h18m3.584890532s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853630877 (156h18m3.584898507s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853960808 (156h18m3.585228438s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854221948 (156h18m3.585489578s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854487648 (156h18m3.585755278s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854706198 (156h18m3.585973818s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854916283 (156h18m3.586183903s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855122541 (156h18m3.586390161s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855323318 (156h18m3.586590939s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855572116 (156h18m3.586839736s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855783854 (156h18m3.587051484s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856043823 (156h18m3.587311442s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856221917 (156h18m3.587489547s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856474260 (156h18m3.587741891s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856684717 (156h18m3.587952347s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856890473 (156h18m3.588158103s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857137107 (156h18m3.588404727s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857407986 (156h18m3.588675616s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.931933679 (156h18m3.663201319s) [upload=162649]: Next returned nil segment
Aug 15 05:44:22.931946533 (156h18m3.663214163s) [upload=162649]: Waiting for error group managing segments...
Aug 15 05:44:22.941125163 (156h18m3.672392803s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.941816452 (156h18m3.673084082s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942102801 (156h18m3.673370441s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942329847 (156h18m3.673597477s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942519253 (156h18m3.673786883s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.944005966 (156h18m3.675273607s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.945343752 (156h18m3.676611392s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.948716731 (156h18m3.679984381s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.954674047 (156h18m3.685941687s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.956262041 (156h18m3.687529671s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.964011115 (156h18m3.695278766s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.973878009 (156h18m3.705145659s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.033905123 (156h18m3.765172773s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039146835 (156h18m3.770414484s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039177702 (156h18m3.770445353s) [seg_pos={0 0}, upload=162649]: Segment reached optimal threshold of 14 pieces.
Aug 15 05:44:23.039726664 (156h18m3.770994314s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHXPzzH8G9xGyABLH7hNnmkZuDdWpoiQkoTxNBzdzNvNQhN29): protocol: read tcp 5.161.89.252:55162->66.23.193.175:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039827213 (156h18m3.771094874s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KNKWRTR6pZfxWFcFwsEKaqGsUMBBSTBsxvmeKsV1GDfEBvQk7): protocol: read tcp 5.161.89.252:18818->79.127.201.215:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039977666 (156h18m3.771245316s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm): protocol: read tcp 5.161.89.252:24186->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.040119473 (156h18m3.771387133s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: context canceled; context canceled
Aug 15 05:44:23.040146273 (156h18m3.771413923s) [seg_pos={0 0}, upload=162649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 15 05:44:23.040241311 (156h18m3.771508962s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 15 05:44:23.040250929 (156h18m3.77151862s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 15 05:44:23.148699216 (156h18m3.879966877s) [upload=162649]: Done uploading segments...
Aug 15 05:44:23.148718513 (156h18m3.879986163s) [upload=162649]: Upload finished. err: <nil>
Duration: 401.450455ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...X......................................................X.] duration:
125.076296ms
FAIL (2025-08-15 06:18:38 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 15 06:17:46.487011506 (156h51m27.218279167s) [upload=163229]: Starting upload Aug 15 06:17:46.487080387 (156h51m27.218348037s) [upload=163229]: Done starting upload Aug 15 06:17:46.487149736 (156h51m27.218417387s) [upload=163229]: Uploading segments... Aug 15 06:17:46.487218095 (156h51m27.218485755s) [upload=163229]: Got next segment. Inline: true Aug 15 06:17:46.487225218 (156h51m27.218492868s) [upload=163229]: Waiting for error group managing segments... Aug 15 06:17:46.487270695 (156h51m27.218538345s) [upload=163229]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 15 06:17:46.487275964 (156h51m27.218543615s) [upload=163229]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 15 06:17:46.487279381 (156h51m27.218547031s) [upload=163229]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 15 06:17:46.490074687 (156h51m27.221342337s) [upload=163229]: Done uploading segments... Aug 15 06:17:46.490086849 (156h51m27.2213545s) [upload=163229]: Upload finished. err: metaclient: Too Many Requests
Duration: 125.098667ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
369.669898ms
FAIL (2025-08-15 06:18:25 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 15 05:44:22.781048516 (156h18m3.512316206s) [upload=162649]: Starting upload
Aug 15 05:44:22.781143685 (156h18m3.512411345s) [upload=162649]: Done starting upload
Aug 15 05:44:22.781205261 (156h18m3.51247291s) [upload=162649]: Uploading segments...
Aug 15 05:44:22.781552763 (156h18m3.512820423s) [upload=162649]: Got next segment. Inline: false
Aug 15 05:44:22.781613949 (156h18m3.512881639s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 15 05:44:22.781623357 (156h18m3.512891007s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 15 05:44:22.853583217 (156h18m3.584850848s) [seg_pos={0 0}, upload=162649]: Begin upload segment...
Aug 15 05:44:22.853614857 (156h18m3.584882487s) [seg_pos={0 0}, upload=162649]: Done begin upload segment.
Aug 15 05:44:22.853622912 (156h18m3.584890532s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853630877 (156h18m3.584898507s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853960808 (156h18m3.585228438s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854221948 (156h18m3.585489578s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854487648 (156h18m3.585755278s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854706198 (156h18m3.585973818s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854916283 (156h18m3.586183903s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855122541 (156h18m3.586390161s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855323318 (156h18m3.586590939s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855572116 (156h18m3.586839736s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855783854 (156h18m3.587051484s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856043823 (156h18m3.587311442s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856221917 (156h18m3.587489547s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856474260 (156h18m3.587741891s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856684717 (156h18m3.587952347s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856890473 (156h18m3.588158103s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857137107 (156h18m3.588404727s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857407986 (156h18m3.588675616s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.931933679 (156h18m3.663201319s) [upload=162649]: Next returned nil segment
Aug 15 05:44:22.931946533 (156h18m3.663214163s) [upload=162649]: Waiting for error group managing segments...
Aug 15 05:44:22.941125163 (156h18m3.672392803s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.941816452 (156h18m3.673084082s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942102801 (156h18m3.673370441s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942329847 (156h18m3.673597477s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942519253 (156h18m3.673786883s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.944005966 (156h18m3.675273607s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.945343752 (156h18m3.676611392s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.948716731 (156h18m3.679984381s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.954674047 (156h18m3.685941687s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.956262041 (156h18m3.687529671s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.964011115 (156h18m3.695278766s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.973878009 (156h18m3.705145659s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.033905123 (156h18m3.765172773s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039146835 (156h18m3.770414484s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039177702 (156h18m3.770445353s) [seg_pos={0 0}, upload=162649]: Segment reached optimal threshold of 14 pieces.
Aug 15 05:44:23.039726664 (156h18m3.770994314s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHXPzzH8G9xGyABLH7hNnmkZuDdWpoiQkoTxNBzdzNvNQhN29): protocol: read tcp 5.161.89.252:55162->66.23.193.175:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039827213 (156h18m3.771094874s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KNKWRTR6pZfxWFcFwsEKaqGsUMBBSTBsxvmeKsV1GDfEBvQk7): protocol: read tcp 5.161.89.252:18818->79.127.201.215:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039977666 (156h18m3.771245316s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm): protocol: read tcp 5.161.89.252:24186->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.040119473 (156h18m3.771387133s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: context canceled; context canceled
Aug 15 05:44:23.040146273 (156h18m3.771413923s) [seg_pos={0 0}, upload=162649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 15 05:44:23.040241311 (156h18m3.771508962s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 15 05:44:23.040250929 (156h18m3.77151862s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 15 05:44:23.148699216 (156h18m3.879966877s) [upload=162649]: Done uploading segments...
Aug 15 05:44:23.148718513 (156h18m3.879986163s) [upload=162649]: Upload finished. err: <nil>
Duration: 369.721234ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [....X......................................................X] duration:
21.565239ms
FAIL (2025-08-15 06:17:46 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 15 06:17:46.487011506 (156h51m27.218279167s) [upload=163229]: Starting upload Aug 15 06:17:46.487080387 (156h51m27.218348037s) [upload=163229]: Done starting upload Aug 15 06:17:46.487149736 (156h51m27.218417387s) [upload=163229]: Uploading segments... Aug 15 06:17:46.487218095 (156h51m27.218485755s) [upload=163229]: Got next segment. Inline: true Aug 15 06:17:46.487225218 (156h51m27.218492868s) [upload=163229]: Waiting for error group managing segments... Aug 15 06:17:46.487270695 (156h51m27.218538345s) [upload=163229]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 15 06:17:46.487275964 (156h51m27.218543615s) [upload=163229]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 15 06:17:46.487279381 (156h51m27.218547031s) [upload=163229]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 15 06:17:46.490074687 (156h51m27.221342337s) [upload=163229]: Done uploading segments... Aug 15 06:17:46.490086849 (156h51m27.2213545s) [upload=163229]: Upload finished. err: metaclient: Too Many Requests
Duration: 21.582382ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
400.621687ms
FAIL (2025-08-15 06:17:16 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 15 05:44:22.781048516 (156h18m3.512316206s) [upload=162649]: Starting upload
Aug 15 05:44:22.781143685 (156h18m3.512411345s) [upload=162649]: Done starting upload
Aug 15 05:44:22.781205261 (156h18m3.51247291s) [upload=162649]: Uploading segments...
Aug 15 05:44:22.781552763 (156h18m3.512820423s) [upload=162649]: Got next segment. Inline: false
Aug 15 05:44:22.781613949 (156h18m3.512881639s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 15 05:44:22.781623357 (156h18m3.512891007s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 15 05:44:22.853583217 (156h18m3.584850848s) [seg_pos={0 0}, upload=162649]: Begin upload segment...
Aug 15 05:44:22.853614857 (156h18m3.584882487s) [seg_pos={0 0}, upload=162649]: Done begin upload segment.
Aug 15 05:44:22.853622912 (156h18m3.584890532s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853630877 (156h18m3.584898507s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853960808 (156h18m3.585228438s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854221948 (156h18m3.585489578s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854487648 (156h18m3.585755278s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854706198 (156h18m3.585973818s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854916283 (156h18m3.586183903s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855122541 (156h18m3.586390161s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855323318 (156h18m3.586590939s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855572116 (156h18m3.586839736s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855783854 (156h18m3.587051484s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856043823 (156h18m3.587311442s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856221917 (156h18m3.587489547s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856474260 (156h18m3.587741891s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856684717 (156h18m3.587952347s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856890473 (156h18m3.588158103s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857137107 (156h18m3.588404727s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857407986 (156h18m3.588675616s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.931933679 (156h18m3.663201319s) [upload=162649]: Next returned nil segment
Aug 15 05:44:22.931946533 (156h18m3.663214163s) [upload=162649]: Waiting for error group managing segments...
Aug 15 05:44:22.941125163 (156h18m3.672392803s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.941816452 (156h18m3.673084082s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942102801 (156h18m3.673370441s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942329847 (156h18m3.673597477s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942519253 (156h18m3.673786883s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.944005966 (156h18m3.675273607s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.945343752 (156h18m3.676611392s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.948716731 (156h18m3.679984381s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.954674047 (156h18m3.685941687s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.956262041 (156h18m3.687529671s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.964011115 (156h18m3.695278766s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.973878009 (156h18m3.705145659s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.033905123 (156h18m3.765172773s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039146835 (156h18m3.770414484s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039177702 (156h18m3.770445353s) [seg_pos={0 0}, upload=162649]: Segment reached optimal threshold of 14 pieces.
Aug 15 05:44:23.039726664 (156h18m3.770994314s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHXPzzH8G9xGyABLH7hNnmkZuDdWpoiQkoTxNBzdzNvNQhN29): protocol: read tcp 5.161.89.252:55162->66.23.193.175:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039827213 (156h18m3.771094874s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KNKWRTR6pZfxWFcFwsEKaqGsUMBBSTBsxvmeKsV1GDfEBvQk7): protocol: read tcp 5.161.89.252:18818->79.127.201.215:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039977666 (156h18m3.771245316s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm): protocol: read tcp 5.161.89.252:24186->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.040119473 (156h18m3.771387133s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: context canceled; context canceled
Aug 15 05:44:23.040146273 (156h18m3.771413923s) [seg_pos={0 0}, upload=162649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 15 05:44:23.040241311 (156h18m3.771508962s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 15 05:44:23.040250929 (156h18m3.77151862s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 15 05:44:23.148699216 (156h18m3.879966877s) [upload=162649]: Done uploading segments...
Aug 15 05:44:23.148718513 (156h18m3.879986163s) [upload=162649]: Upload finished. err: <nil>
Duration: 400.641764ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
124.92836ms
FAIL (2025-08-15 06:17:13 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 15 05:39:32.377895512 (156h13m13.109163133s) [upload=162565]: Starting upload Aug 15 05:39:32.377934446 (156h13m13.109202066s) [upload=162565]: Done starting upload Aug 15 05:39:32.377970092 (156h13m13.109237722s) [upload=162565]: Uploading segments... Aug 15 05:39:32.378004016 (156h13m13.109271646s) [upload=162565]: Got next segment. Inline: true Aug 15 05:39:32.378008174 (156h13m13.109275794s) [upload=162565]: Waiting for error group managing segments... Aug 15 05:39:32.378039042 (156h13m13.109306672s) [upload=162565]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 15 05:39:32.378042799 (156h13m13.109310429s) [upload=162565]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 15 05:39:32.378045124 (156h13m13.109312754s) [upload=162565]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 15 05:39:32.380223698 (156h13m13.111491318s) [upload=162565]: Done uploading segments... Aug 15 05:39:32.380230541 (156h13m13.111498161s) [upload=162565]: Upload finished. err: metaclient: Too Many Requests
Duration: 124.95493ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
374.581132ms
FAIL (2025-08-15 06:16:50 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 15 05:39:32.377895512 (156h13m13.109163133s) [upload=162565]: Starting upload Aug 15 05:39:32.377934446 (156h13m13.109202066s) [upload=162565]: Done starting upload Aug 15 05:39:32.377970092 (156h13m13.109237722s) [upload=162565]: Uploading segments... Aug 15 05:39:32.378004016 (156h13m13.109271646s) [upload=162565]: Got next segment. Inline: true Aug 15 05:39:32.378008174 (156h13m13.109275794s) [upload=162565]: Waiting for error group managing segments... Aug 15 05:39:32.378039042 (156h13m13.109306672s) [upload=162565]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 15 05:39:32.378042799 (156h13m13.109310429s) [upload=162565]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 15 05:39:32.378045124 (156h13m13.109312754s) [upload=162565]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 15 05:39:32.380223698 (156h13m13.111491318s) [upload=162565]: Done uploading segments... Aug 15 05:39:32.380230541 (156h13m13.111498161s) [upload=162565]: Upload finished. err: metaclient: Too Many Requests
Duration: 374.599357ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
302.161898ms
FAIL (2025-08-15 06:16:29 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 15 05:44:22.781048516 (156h18m3.512316206s) [upload=162649]: Starting upload
Aug 15 05:44:22.781143685 (156h18m3.512411345s) [upload=162649]: Done starting upload
Aug 15 05:44:22.781205261 (156h18m3.51247291s) [upload=162649]: Uploading segments...
Aug 15 05:44:22.781552763 (156h18m3.512820423s) [upload=162649]: Got next segment. Inline: false
Aug 15 05:44:22.781613949 (156h18m3.512881639s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 15 05:44:22.781623357 (156h18m3.512891007s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 15 05:44:22.853583217 (156h18m3.584850848s) [seg_pos={0 0}, upload=162649]: Begin upload segment...
Aug 15 05:44:22.853614857 (156h18m3.584882487s) [seg_pos={0 0}, upload=162649]: Done begin upload segment.
Aug 15 05:44:22.853622912 (156h18m3.584890532s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853630877 (156h18m3.584898507s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853960808 (156h18m3.585228438s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854221948 (156h18m3.585489578s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854487648 (156h18m3.585755278s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854706198 (156h18m3.585973818s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854916283 (156h18m3.586183903s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855122541 (156h18m3.586390161s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855323318 (156h18m3.586590939s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855572116 (156h18m3.586839736s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855783854 (156h18m3.587051484s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856043823 (156h18m3.587311442s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856221917 (156h18m3.587489547s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856474260 (156h18m3.587741891s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856684717 (156h18m3.587952347s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856890473 (156h18m3.588158103s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857137107 (156h18m3.588404727s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857407986 (156h18m3.588675616s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.931933679 (156h18m3.663201319s) [upload=162649]: Next returned nil segment
Aug 15 05:44:22.931946533 (156h18m3.663214163s) [upload=162649]: Waiting for error group managing segments...
Aug 15 05:44:22.941125163 (156h18m3.672392803s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.941816452 (156h18m3.673084082s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942102801 (156h18m3.673370441s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942329847 (156h18m3.673597477s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942519253 (156h18m3.673786883s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.944005966 (156h18m3.675273607s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.945343752 (156h18m3.676611392s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.948716731 (156h18m3.679984381s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.954674047 (156h18m3.685941687s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.956262041 (156h18m3.687529671s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.964011115 (156h18m3.695278766s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.973878009 (156h18m3.705145659s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.033905123 (156h18m3.765172773s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039146835 (156h18m3.770414484s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039177702 (156h18m3.770445353s) [seg_pos={0 0}, upload=162649]: Segment reached optimal threshold of 14 pieces.
Aug 15 05:44:23.039726664 (156h18m3.770994314s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHXPzzH8G9xGyABLH7hNnmkZuDdWpoiQkoTxNBzdzNvNQhN29): protocol: read tcp 5.161.89.252:55162->66.23.193.175:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039827213 (156h18m3.771094874s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KNKWRTR6pZfxWFcFwsEKaqGsUMBBSTBsxvmeKsV1GDfEBvQk7): protocol: read tcp 5.161.89.252:18818->79.127.201.215:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039977666 (156h18m3.771245316s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm): protocol: read tcp 5.161.89.252:24186->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.040119473 (156h18m3.771387133s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: context canceled; context canceled
Aug 15 05:44:23.040146273 (156h18m3.771413923s) [seg_pos={0 0}, upload=162649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 15 05:44:23.040241311 (156h18m3.771508962s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 15 05:44:23.040250929 (156h18m3.77151862s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 15 05:44:23.148699216 (156h18m3.879966877s) [upload=162649]: Done uploading segments...
Aug 15 05:44:23.148718513 (156h18m3.879986163s) [upload=162649]: Upload finished. err: <nil>
Duration: 302.174242ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
128.615501ms
FAIL (2025-08-15 06:16:27 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 15 05:39:32.377895512 (156h13m13.109163133s) [upload=162565]: Starting upload Aug 15 05:39:32.377934446 (156h13m13.109202066s) [upload=162565]: Done starting upload Aug 15 05:39:32.377970092 (156h13m13.109237722s) [upload=162565]: Uploading segments... Aug 15 05:39:32.378004016 (156h13m13.109271646s) [upload=162565]: Got next segment. Inline: true Aug 15 05:39:32.378008174 (156h13m13.109275794s) [upload=162565]: Waiting for error group managing segments... Aug 15 05:39:32.378039042 (156h13m13.109306672s) [upload=162565]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 15 05:39:32.378042799 (156h13m13.109310429s) [upload=162565]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 15 05:39:32.378045124 (156h13m13.109312754s) [upload=162565]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 15 05:39:32.380223698 (156h13m13.111491318s) [upload=162565]: Done uploading segments... Aug 15 05:39:32.380230541 (156h13m13.111498161s) [upload=162565]: Upload finished. err: metaclient: Too Many Requests
Duration: 128.635758ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
224.879471ms
FAIL (2025-08-15 06:16:07 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 224.91055ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
403.532919ms
FAIL (2025-08-15 06:15:23 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 15 05:44:22.781048516 (156h18m3.512316206s) [upload=162649]: Starting upload
Aug 15 05:44:22.781143685 (156h18m3.512411345s) [upload=162649]: Done starting upload
Aug 15 05:44:22.781205261 (156h18m3.51247291s) [upload=162649]: Uploading segments...
Aug 15 05:44:22.781552763 (156h18m3.512820423s) [upload=162649]: Got next segment. Inline: false
Aug 15 05:44:22.781613949 (156h18m3.512881639s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 15 05:44:22.781623357 (156h18m3.512891007s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 15 05:44:22.853583217 (156h18m3.584850848s) [seg_pos={0 0}, upload=162649]: Begin upload segment...
Aug 15 05:44:22.853614857 (156h18m3.584882487s) [seg_pos={0 0}, upload=162649]: Done begin upload segment.
Aug 15 05:44:22.853622912 (156h18m3.584890532s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853630877 (156h18m3.584898507s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853960808 (156h18m3.585228438s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854221948 (156h18m3.585489578s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854487648 (156h18m3.585755278s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854706198 (156h18m3.585973818s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854916283 (156h18m3.586183903s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855122541 (156h18m3.586390161s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855323318 (156h18m3.586590939s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855572116 (156h18m3.586839736s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855783854 (156h18m3.587051484s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856043823 (156h18m3.587311442s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856221917 (156h18m3.587489547s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856474260 (156h18m3.587741891s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856684717 (156h18m3.587952347s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856890473 (156h18m3.588158103s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857137107 (156h18m3.588404727s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857407986 (156h18m3.588675616s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.931933679 (156h18m3.663201319s) [upload=162649]: Next returned nil segment
Aug 15 05:44:22.931946533 (156h18m3.663214163s) [upload=162649]: Waiting for error group managing segments...
Aug 15 05:44:22.941125163 (156h18m3.672392803s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.941816452 (156h18m3.673084082s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942102801 (156h18m3.673370441s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942329847 (156h18m3.673597477s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942519253 (156h18m3.673786883s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.944005966 (156h18m3.675273607s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.945343752 (156h18m3.676611392s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.948716731 (156h18m3.679984381s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.954674047 (156h18m3.685941687s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.956262041 (156h18m3.687529671s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.964011115 (156h18m3.695278766s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.973878009 (156h18m3.705145659s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.033905123 (156h18m3.765172773s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039146835 (156h18m3.770414484s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039177702 (156h18m3.770445353s) [seg_pos={0 0}, upload=162649]: Segment reached optimal threshold of 14 pieces.
Aug 15 05:44:23.039726664 (156h18m3.770994314s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHXPzzH8G9xGyABLH7hNnmkZuDdWpoiQkoTxNBzdzNvNQhN29): protocol: read tcp 5.161.89.252:55162->66.23.193.175:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039827213 (156h18m3.771094874s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KNKWRTR6pZfxWFcFwsEKaqGsUMBBSTBsxvmeKsV1GDfEBvQk7): protocol: read tcp 5.161.89.252:18818->79.127.201.215:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039977666 (156h18m3.771245316s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm): protocol: read tcp 5.161.89.252:24186->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.040119473 (156h18m3.771387133s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: context canceled; context canceled
Aug 15 05:44:23.040146273 (156h18m3.771413923s) [seg_pos={0 0}, upload=162649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 15 05:44:23.040241311 (156h18m3.771508962s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 15 05:44:23.040250929 (156h18m3.77151862s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 15 05:44:23.148699216 (156h18m3.879966877s) [upload=162649]: Done uploading segments...
Aug 15 05:44:23.148718513 (156h18m3.879986163s) [upload=162649]: Upload finished. err: <nil>
Duration: 403.554851ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
113.595588ms
FAIL (2025-08-15 06:15:21 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 15 05:39:32.377895512 (156h13m13.109163133s) [upload=162565]: Starting upload Aug 15 05:39:32.377934446 (156h13m13.109202066s) [upload=162565]: Done starting upload Aug 15 05:39:32.377970092 (156h13m13.109237722s) [upload=162565]: Uploading segments... Aug 15 05:39:32.378004016 (156h13m13.109271646s) [upload=162565]: Got next segment. Inline: true Aug 15 05:39:32.378008174 (156h13m13.109275794s) [upload=162565]: Waiting for error group managing segments... Aug 15 05:39:32.378039042 (156h13m13.109306672s) [upload=162565]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 15 05:39:32.378042799 (156h13m13.109310429s) [upload=162565]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 15 05:39:32.378045124 (156h13m13.109312754s) [upload=162565]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 15 05:39:32.380223698 (156h13m13.111491318s) [upload=162565]: Done uploading segments... Aug 15 05:39:32.380230541 (156h13m13.111498161s) [upload=162565]: Upload finished. err: metaclient: Too Many Requests
Duration: 113.613322ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
187.077429ms
FAIL (2025-08-15 06:14:54 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 187.122704ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
329.18406ms
FAIL (2025-08-15 06:14:46 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 15 05:44:22.781048516 (156h18m3.512316206s) [upload=162649]: Starting upload
Aug 15 05:44:22.781143685 (156h18m3.512411345s) [upload=162649]: Done starting upload
Aug 15 05:44:22.781205261 (156h18m3.51247291s) [upload=162649]: Uploading segments...
Aug 15 05:44:22.781552763 (156h18m3.512820423s) [upload=162649]: Got next segment. Inline: false
Aug 15 05:44:22.781613949 (156h18m3.512881639s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 15 05:44:22.781623357 (156h18m3.512891007s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 15 05:44:22.853583217 (156h18m3.584850848s) [seg_pos={0 0}, upload=162649]: Begin upload segment...
Aug 15 05:44:22.853614857 (156h18m3.584882487s) [seg_pos={0 0}, upload=162649]: Done begin upload segment.
Aug 15 05:44:22.853622912 (156h18m3.584890532s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853630877 (156h18m3.584898507s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853960808 (156h18m3.585228438s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854221948 (156h18m3.585489578s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854487648 (156h18m3.585755278s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854706198 (156h18m3.585973818s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854916283 (156h18m3.586183903s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855122541 (156h18m3.586390161s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855323318 (156h18m3.586590939s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855572116 (156h18m3.586839736s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855783854 (156h18m3.587051484s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856043823 (156h18m3.587311442s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856221917 (156h18m3.587489547s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856474260 (156h18m3.587741891s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856684717 (156h18m3.587952347s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856890473 (156h18m3.588158103s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857137107 (156h18m3.588404727s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857407986 (156h18m3.588675616s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.931933679 (156h18m3.663201319s) [upload=162649]: Next returned nil segment
Aug 15 05:44:22.931946533 (156h18m3.663214163s) [upload=162649]: Waiting for error group managing segments...
Aug 15 05:44:22.941125163 (156h18m3.672392803s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.941816452 (156h18m3.673084082s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942102801 (156h18m3.673370441s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942329847 (156h18m3.673597477s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942519253 (156h18m3.673786883s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.944005966 (156h18m3.675273607s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.945343752 (156h18m3.676611392s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.948716731 (156h18m3.679984381s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.954674047 (156h18m3.685941687s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.956262041 (156h18m3.687529671s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.964011115 (156h18m3.695278766s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.973878009 (156h18m3.705145659s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.033905123 (156h18m3.765172773s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039146835 (156h18m3.770414484s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039177702 (156h18m3.770445353s) [seg_pos={0 0}, upload=162649]: Segment reached optimal threshold of 14 pieces.
Aug 15 05:44:23.039726664 (156h18m3.770994314s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHXPzzH8G9xGyABLH7hNnmkZuDdWpoiQkoTxNBzdzNvNQhN29): protocol: read tcp 5.161.89.252:55162->66.23.193.175:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039827213 (156h18m3.771094874s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KNKWRTR6pZfxWFcFwsEKaqGsUMBBSTBsxvmeKsV1GDfEBvQk7): protocol: read tcp 5.161.89.252:18818->79.127.201.215:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039977666 (156h18m3.771245316s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm): protocol: read tcp 5.161.89.252:24186->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.040119473 (156h18m3.771387133s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: context canceled; context canceled
Aug 15 05:44:23.040146273 (156h18m3.771413923s) [seg_pos={0 0}, upload=162649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 15 05:44:23.040241311 (156h18m3.771508962s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 15 05:44:23.040250929 (156h18m3.77151862s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 15 05:44:23.148699216 (156h18m3.879966877s) [upload=162649]: Done uploading segments...
Aug 15 05:44:23.148718513 (156h18m3.879986163s) [upload=162649]: Upload finished. err: <nil>
Duration: 329.204268ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
118.447646ms
FAIL (2025-08-15 06:14:22 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 15 05:39:32.377895512 (156h13m13.109163133s) [upload=162565]: Starting upload Aug 15 05:39:32.377934446 (156h13m13.109202066s) [upload=162565]: Done starting upload Aug 15 05:39:32.377970092 (156h13m13.109237722s) [upload=162565]: Uploading segments... Aug 15 05:39:32.378004016 (156h13m13.109271646s) [upload=162565]: Got next segment. Inline: true Aug 15 05:39:32.378008174 (156h13m13.109275794s) [upload=162565]: Waiting for error group managing segments... Aug 15 05:39:32.378039042 (156h13m13.109306672s) [upload=162565]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 15 05:39:32.378042799 (156h13m13.109310429s) [upload=162565]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 15 05:39:32.378045124 (156h13m13.109312754s) [upload=162565]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 15 05:39:32.380223698 (156h13m13.111491318s) [upload=162565]: Done uploading segments... Aug 15 05:39:32.380230541 (156h13m13.111498161s) [upload=162565]: Upload finished. err: metaclient: Too Many Requests
Duration: 118.468065ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
402.040375ms
FAIL (2025-08-15 06:14:15 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 402.06463ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
193.370964ms
FAIL (2025-08-15 06:13:57 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 193.393217ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
323.301217ms
FAIL (2025-08-15 06:13:52 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 15 05:44:22.781048516 (156h18m3.512316206s) [upload=162649]: Starting upload
Aug 15 05:44:22.781143685 (156h18m3.512411345s) [upload=162649]: Done starting upload
Aug 15 05:44:22.781205261 (156h18m3.51247291s) [upload=162649]: Uploading segments...
Aug 15 05:44:22.781552763 (156h18m3.512820423s) [upload=162649]: Got next segment. Inline: false
Aug 15 05:44:22.781613949 (156h18m3.512881639s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 15 05:44:22.781623357 (156h18m3.512891007s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 15 05:44:22.853583217 (156h18m3.584850848s) [seg_pos={0 0}, upload=162649]: Begin upload segment...
Aug 15 05:44:22.853614857 (156h18m3.584882487s) [seg_pos={0 0}, upload=162649]: Done begin upload segment.
Aug 15 05:44:22.853622912 (156h18m3.584890532s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853630877 (156h18m3.584898507s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853960808 (156h18m3.585228438s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854221948 (156h18m3.585489578s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854487648 (156h18m3.585755278s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854706198 (156h18m3.585973818s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854916283 (156h18m3.586183903s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855122541 (156h18m3.586390161s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855323318 (156h18m3.586590939s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855572116 (156h18m3.586839736s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855783854 (156h18m3.587051484s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856043823 (156h18m3.587311442s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856221917 (156h18m3.587489547s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856474260 (156h18m3.587741891s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856684717 (156h18m3.587952347s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856890473 (156h18m3.588158103s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857137107 (156h18m3.588404727s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857407986 (156h18m3.588675616s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.931933679 (156h18m3.663201319s) [upload=162649]: Next returned nil segment
Aug 15 05:44:22.931946533 (156h18m3.663214163s) [upload=162649]: Waiting for error group managing segments...
Aug 15 05:44:22.941125163 (156h18m3.672392803s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.941816452 (156h18m3.673084082s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942102801 (156h18m3.673370441s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942329847 (156h18m3.673597477s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942519253 (156h18m3.673786883s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.944005966 (156h18m3.675273607s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.945343752 (156h18m3.676611392s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.948716731 (156h18m3.679984381s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.954674047 (156h18m3.685941687s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.956262041 (156h18m3.687529671s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.964011115 (156h18m3.695278766s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.973878009 (156h18m3.705145659s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.033905123 (156h18m3.765172773s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039146835 (156h18m3.770414484s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039177702 (156h18m3.770445353s) [seg_pos={0 0}, upload=162649]: Segment reached optimal threshold of 14 pieces.
Aug 15 05:44:23.039726664 (156h18m3.770994314s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHXPzzH8G9xGyABLH7hNnmkZuDdWpoiQkoTxNBzdzNvNQhN29): protocol: read tcp 5.161.89.252:55162->66.23.193.175:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039827213 (156h18m3.771094874s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KNKWRTR6pZfxWFcFwsEKaqGsUMBBSTBsxvmeKsV1GDfEBvQk7): protocol: read tcp 5.161.89.252:18818->79.127.201.215:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039977666 (156h18m3.771245316s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm): protocol: read tcp 5.161.89.252:24186->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.040119473 (156h18m3.771387133s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: context canceled; context canceled
Aug 15 05:44:23.040146273 (156h18m3.771413923s) [seg_pos={0 0}, upload=162649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 15 05:44:23.040241311 (156h18m3.771508962s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 15 05:44:23.040250929 (156h18m3.77151862s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 15 05:44:23.148699216 (156h18m3.879966877s) [upload=162649]: Done uploading segments...
Aug 15 05:44:23.148718513 (156h18m3.879986163s) [upload=162649]: Upload finished. err: <nil>
Duration: 323.323008ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
135.589988ms
FAIL (2025-08-15 06:13:51 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 15 05:39:32.377895512 (156h13m13.109163133s) [upload=162565]: Starting upload Aug 15 05:39:32.377934446 (156h13m13.109202066s) [upload=162565]: Done starting upload Aug 15 05:39:32.377970092 (156h13m13.109237722s) [upload=162565]: Uploading segments... Aug 15 05:39:32.378004016 (156h13m13.109271646s) [upload=162565]: Got next segment. Inline: true Aug 15 05:39:32.378008174 (156h13m13.109275794s) [upload=162565]: Waiting for error group managing segments... Aug 15 05:39:32.378039042 (156h13m13.109306672s) [upload=162565]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 15 05:39:32.378042799 (156h13m13.109310429s) [upload=162565]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 15 05:39:32.378045124 (156h13m13.109312754s) [upload=162565]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 15 05:39:32.380223698 (156h13m13.111491318s) [upload=162565]: Done uploading segments... Aug 15 05:39:32.380230541 (156h13m13.111498161s) [upload=162565]: Upload finished. err: metaclient: Too Many Requests
Duration: 135.616579ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
248.011827ms
FAIL (2025-08-15 06:13:40 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 248.036894ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
418.556539ms
FAIL (2025-08-15 06:12:51 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 418.57836ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
352.08545ms
FAIL (2025-08-15 06:12:48 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 15 05:44:22.781048516 (156h18m3.512316206s) [upload=162649]: Starting upload
Aug 15 05:44:22.781143685 (156h18m3.512411345s) [upload=162649]: Done starting upload
Aug 15 05:44:22.781205261 (156h18m3.51247291s) [upload=162649]: Uploading segments...
Aug 15 05:44:22.781552763 (156h18m3.512820423s) [upload=162649]: Got next segment. Inline: false
Aug 15 05:44:22.781613949 (156h18m3.512881639s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 15 05:44:22.781623357 (156h18m3.512891007s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 15 05:44:22.853583217 (156h18m3.584850848s) [seg_pos={0 0}, upload=162649]: Begin upload segment...
Aug 15 05:44:22.853614857 (156h18m3.584882487s) [seg_pos={0 0}, upload=162649]: Done begin upload segment.
Aug 15 05:44:22.853622912 (156h18m3.584890532s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853630877 (156h18m3.584898507s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853960808 (156h18m3.585228438s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854221948 (156h18m3.585489578s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854487648 (156h18m3.585755278s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854706198 (156h18m3.585973818s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854916283 (156h18m3.586183903s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855122541 (156h18m3.586390161s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855323318 (156h18m3.586590939s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855572116 (156h18m3.586839736s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855783854 (156h18m3.587051484s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856043823 (156h18m3.587311442s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856221917 (156h18m3.587489547s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856474260 (156h18m3.587741891s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856684717 (156h18m3.587952347s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856890473 (156h18m3.588158103s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857137107 (156h18m3.588404727s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857407986 (156h18m3.588675616s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.931933679 (156h18m3.663201319s) [upload=162649]: Next returned nil segment
Aug 15 05:44:22.931946533 (156h18m3.663214163s) [upload=162649]: Waiting for error group managing segments...
Aug 15 05:44:22.941125163 (156h18m3.672392803s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.941816452 (156h18m3.673084082s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942102801 (156h18m3.673370441s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942329847 (156h18m3.673597477s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942519253 (156h18m3.673786883s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.944005966 (156h18m3.675273607s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.945343752 (156h18m3.676611392s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.948716731 (156h18m3.679984381s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.954674047 (156h18m3.685941687s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.956262041 (156h18m3.687529671s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.964011115 (156h18m3.695278766s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.973878009 (156h18m3.705145659s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.033905123 (156h18m3.765172773s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039146835 (156h18m3.770414484s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039177702 (156h18m3.770445353s) [seg_pos={0 0}, upload=162649]: Segment reached optimal threshold of 14 pieces.
Aug 15 05:44:23.039726664 (156h18m3.770994314s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHXPzzH8G9xGyABLH7hNnmkZuDdWpoiQkoTxNBzdzNvNQhN29): protocol: read tcp 5.161.89.252:55162->66.23.193.175:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039827213 (156h18m3.771094874s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KNKWRTR6pZfxWFcFwsEKaqGsUMBBSTBsxvmeKsV1GDfEBvQk7): protocol: read tcp 5.161.89.252:18818->79.127.201.215:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039977666 (156h18m3.771245316s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm): protocol: read tcp 5.161.89.252:24186->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.040119473 (156h18m3.771387133s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: context canceled; context canceled
Aug 15 05:44:23.040146273 (156h18m3.771413923s) [seg_pos={0 0}, upload=162649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 15 05:44:23.040241311 (156h18m3.771508962s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 15 05:44:23.040250929 (156h18m3.77151862s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 15 05:44:23.148699216 (156h18m3.879966877s) [upload=162649]: Done uploading segments...
Aug 15 05:44:23.148718513 (156h18m3.879986163s) [upload=162649]: Upload finished. err: <nil>
Duration: 352.106059ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
128.277094ms
FAIL (2025-08-15 06:12:37 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 15 05:39:32.377895512 (156h13m13.109163133s) [upload=162565]: Starting upload Aug 15 05:39:32.377934446 (156h13m13.109202066s) [upload=162565]: Done starting upload Aug 15 05:39:32.377970092 (156h13m13.109237722s) [upload=162565]: Uploading segments... Aug 15 05:39:32.378004016 (156h13m13.109271646s) [upload=162565]: Got next segment. Inline: true Aug 15 05:39:32.378008174 (156h13m13.109275794s) [upload=162565]: Waiting for error group managing segments... Aug 15 05:39:32.378039042 (156h13m13.109306672s) [upload=162565]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 15 05:39:32.378042799 (156h13m13.109310429s) [upload=162565]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 15 05:39:32.378045124 (156h13m13.109312754s) [upload=162565]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 15 05:39:32.380223698 (156h13m13.111491318s) [upload=162565]: Done uploading segments... Aug 15 05:39:32.380230541 (156h13m13.111498161s) [upload=162565]: Upload finished. err: metaclient: Too Many Requests
Duration: 128.29558ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
737.278207ms
FAIL (2025-08-15 06:12:24 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 737.390459ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
323.062135ms
FAIL (2025-08-15 06:11:57 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 15 05:44:22.781048516 (156h18m3.512316206s) [upload=162649]: Starting upload
Aug 15 05:44:22.781143685 (156h18m3.512411345s) [upload=162649]: Done starting upload
Aug 15 05:44:22.781205261 (156h18m3.51247291s) [upload=162649]: Uploading segments...
Aug 15 05:44:22.781552763 (156h18m3.512820423s) [upload=162649]: Got next segment. Inline: false
Aug 15 05:44:22.781613949 (156h18m3.512881639s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 15 05:44:22.781623357 (156h18m3.512891007s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 15 05:44:22.853583217 (156h18m3.584850848s) [seg_pos={0 0}, upload=162649]: Begin upload segment...
Aug 15 05:44:22.853614857 (156h18m3.584882487s) [seg_pos={0 0}, upload=162649]: Done begin upload segment.
Aug 15 05:44:22.853622912 (156h18m3.584890532s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853630877 (156h18m3.584898507s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853960808 (156h18m3.585228438s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854221948 (156h18m3.585489578s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854487648 (156h18m3.585755278s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854706198 (156h18m3.585973818s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854916283 (156h18m3.586183903s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855122541 (156h18m3.586390161s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855323318 (156h18m3.586590939s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855572116 (156h18m3.586839736s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855783854 (156h18m3.587051484s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856043823 (156h18m3.587311442s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856221917 (156h18m3.587489547s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856474260 (156h18m3.587741891s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856684717 (156h18m3.587952347s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856890473 (156h18m3.588158103s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857137107 (156h18m3.588404727s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857407986 (156h18m3.588675616s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.931933679 (156h18m3.663201319s) [upload=162649]: Next returned nil segment
Aug 15 05:44:22.931946533 (156h18m3.663214163s) [upload=162649]: Waiting for error group managing segments...
Aug 15 05:44:22.941125163 (156h18m3.672392803s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.941816452 (156h18m3.673084082s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942102801 (156h18m3.673370441s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942329847 (156h18m3.673597477s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942519253 (156h18m3.673786883s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.944005966 (156h18m3.675273607s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.945343752 (156h18m3.676611392s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.948716731 (156h18m3.679984381s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.954674047 (156h18m3.685941687s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.956262041 (156h18m3.687529671s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.964011115 (156h18m3.695278766s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.973878009 (156h18m3.705145659s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.033905123 (156h18m3.765172773s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039146835 (156h18m3.770414484s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039177702 (156h18m3.770445353s) [seg_pos={0 0}, upload=162649]: Segment reached optimal threshold of 14 pieces.
Aug 15 05:44:23.039726664 (156h18m3.770994314s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHXPzzH8G9xGyABLH7hNnmkZuDdWpoiQkoTxNBzdzNvNQhN29): protocol: read tcp 5.161.89.252:55162->66.23.193.175:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039827213 (156h18m3.771094874s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KNKWRTR6pZfxWFcFwsEKaqGsUMBBSTBsxvmeKsV1GDfEBvQk7): protocol: read tcp 5.161.89.252:18818->79.127.201.215:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039977666 (156h18m3.771245316s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm): protocol: read tcp 5.161.89.252:24186->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.040119473 (156h18m3.771387133s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: context canceled; context canceled
Aug 15 05:44:23.040146273 (156h18m3.771413923s) [seg_pos={0 0}, upload=162649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 15 05:44:23.040241311 (156h18m3.771508962s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 15 05:44:23.040250929 (156h18m3.77151862s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 15 05:44:23.148699216 (156h18m3.879966877s) [upload=162649]: Done uploading segments...
Aug 15 05:44:23.148718513 (156h18m3.879986163s) [upload=162649]: Upload finished. err: <nil>
Duration: 323.080991ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
281.446955ms
FAIL (2025-08-15 06:11:52 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 281.48087ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
116.726578ms
FAIL (2025-08-15 06:11:31 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 15 05:39:32.377895512 (156h13m13.109163133s) [upload=162565]: Starting upload Aug 15 05:39:32.377934446 (156h13m13.109202066s) [upload=162565]: Done starting upload Aug 15 05:39:32.377970092 (156h13m13.109237722s) [upload=162565]: Uploading segments... Aug 15 05:39:32.378004016 (156h13m13.109271646s) [upload=162565]: Got next segment. Inline: true Aug 15 05:39:32.378008174 (156h13m13.109275794s) [upload=162565]: Waiting for error group managing segments... Aug 15 05:39:32.378039042 (156h13m13.109306672s) [upload=162565]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 15 05:39:32.378042799 (156h13m13.109310429s) [upload=162565]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 15 05:39:32.378045124 (156h13m13.109312754s) [upload=162565]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 15 05:39:32.380223698 (156h13m13.111491318s) [upload=162565]: Done uploading segments... Aug 15 05:39:32.380230541 (156h13m13.111498161s) [upload=162565]: Upload finished. err: metaclient: Too Many Requests
Duration: 116.745073ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
639.060264ms
FAIL (2025-08-15 06:11:26 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 15 05:44:22.781048516 (156h18m3.512316206s) [upload=162649]: Starting upload
Aug 15 05:44:22.781143685 (156h18m3.512411345s) [upload=162649]: Done starting upload
Aug 15 05:44:22.781205261 (156h18m3.51247291s) [upload=162649]: Uploading segments...
Aug 15 05:44:22.781552763 (156h18m3.512820423s) [upload=162649]: Got next segment. Inline: false
Aug 15 05:44:22.781613949 (156h18m3.512881639s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 15 05:44:22.781623357 (156h18m3.512891007s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 15 05:44:22.853583217 (156h18m3.584850848s) [seg_pos={0 0}, upload=162649]: Begin upload segment...
Aug 15 05:44:22.853614857 (156h18m3.584882487s) [seg_pos={0 0}, upload=162649]: Done begin upload segment.
Aug 15 05:44:22.853622912 (156h18m3.584890532s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853630877 (156h18m3.584898507s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.853960808 (156h18m3.585228438s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854221948 (156h18m3.585489578s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854487648 (156h18m3.585755278s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854706198 (156h18m3.585973818s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.854916283 (156h18m3.586183903s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855122541 (156h18m3.586390161s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855323318 (156h18m3.586590939s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855572116 (156h18m3.586839736s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.855783854 (156h18m3.587051484s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856043823 (156h18m3.587311442s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856221917 (156h18m3.587489547s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856474260 (156h18m3.587741891s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856684717 (156h18m3.587952347s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.856890473 (156h18m3.588158103s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857137107 (156h18m3.588404727s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.857407986 (156h18m3.588675616s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Uploading piece...
Aug 15 05:44:22.931933679 (156h18m3.663201319s) [upload=162649]: Next returned nil segment
Aug 15 05:44:22.931946533 (156h18m3.663214163s) [upload=162649]: Waiting for error group managing segments...
Aug 15 05:44:22.941125163 (156h18m3.672392803s) [noise=true , address=79.127.219.48:4086 , piece_id=VKG6N5XZTR4AX2HXWXC57G2GNBUFITXGDQYM5E2625KKQQN33FEA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.941816452 (156h18m3.673084082s) [noise=true , address=77.90.31.16:4762 , piece_id=3BSD43Y2NPPCH5NNXCAISZHU6PHBJEMPXNPRGF3DJHA7RIBGFX5A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942102801 (156h18m3.673370441s) [noise=true , address=77.90.31.18:4824 , piece_id=H5WNXV4IOB567DYGTZGVHM2CQNEEJBZDALG6ZTCY5FGIBNGZSVVQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942329847 (156h18m3.673597477s) [noise=true , address=77.90.31.15:4706 , piece_id=5UAWSBNKIKQJAYLKUBRH475KL66UXN4JGPZTXEIZWPO7HKAUHWMA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.942519253 (156h18m3.673786883s) [noise=true , address=77.90.31.17:4773 , piece_id=P6M43KJPMP7BUC2AMIILM6WU33WDPSECP3IUKUCY6ZSQGCXCXVMQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.944005966 (156h18m3.675273607s) [noise=true , address=77.90.31.10:4524 , piece_id=ZW3IR22TKWSQ4MXJIDKFLULP7IE2ESI7YZD7WP4YIXAXNKSWJ5JA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.945343752 (156h18m3.676611392s) [noise=true , address=104.36.82.8:4998 , piece_id=ULREOZBR3QPLFMLBFHLWB23HFP3SGPLVNOZPTSOTCO5MLMUT5Z6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.948716731 (156h18m3.679984381s) [noise=true , address=45.91.116.10:4111 , piece_id=Z3MBGP7HZNNL7HZHCSH3BL3FNYO3IUZVB6JZSREWI276FGIKYKCA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.954674047 (156h18m3.685941687s) [noise=true , address=104.36.82.44:4901 , piece_id=WLH2MT3QCZ6DTK2V3WUUJMIISWGKEZ75ZLG7N2IH6GNY6AJZ52UA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.956262041 (156h18m3.687529671s) [noise=true , address=104.36.82.34:4564 , piece_id=KNQGA4CSDOEADMJL7PJ2R53U26ZBBF2AHFI7XP3Z5NOMSBHKWXPA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.964011115 (156h18m3.695278766s) [noise=true , address=104.36.82.58:4285 , piece_id=4M25B3SJLJTJZGDT4PN7QP3RFYD43AZOWFHLWZ32R6CICBTNXXOQ, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:22.973878009 (156h18m3.705145659s) [noise=true , address=104.36.82.5:4871 , piece_id=7QNZRRJLZPZV3XNGVOBNUVXIRGQ6P6TX6BAHWQ5S4HWZDE3KLY6A, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.033905123 (156h18m3.765172773s) [noise=true , address=104.36.82.19:4407 , piece_id=54EIO4D7MBXPAOJNOGAIVFDUBJ6CWMDDVXW3AONPKOZUNJNZ6F5Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039146835 (156h18m3.770414484s) [noise=true , address=104.36.82.23:4452 , piece_id=TY55FDIBHHDRHBSAXOI7ULAHQYKWFLCJNNHH6XTBAKH5LWZJRRZA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: <nil>
Aug 15 05:44:23.039177702 (156h18m3.770445353s) [seg_pos={0 0}, upload=162649]: Segment reached optimal threshold of 14 pieces.
Aug 15 05:44:23.039726664 (156h18m3.770994314s) [noise=true , address=66.23.193.175:4556 , piece_id=AXN3RJB5JCKKXQGLBSQPWVRUYJUWF4NHJGDHQ5NXTZOJCM2RYXWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHXPzzH8G9xGyABLH7hNnmkZuDdWpoiQkoTxNBzdzNvNQhN29): protocol: read tcp 5.161.89.252:55162->66.23.193.175:4556: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039827213 (156h18m3.771094874s) [noise=true , address=79.127.201.215:4352 , piece_id=OFKFZPWSSOUE6DL2LZ3V3RSNRJSROTYWTQH2ADIR7URJFO2QNZWA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KNKWRTR6pZfxWFcFwsEKaqGsUMBBSTBsxvmeKsV1GDfEBvQk7): protocol: read tcp 5.161.89.252:18818->79.127.201.215:4352: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.039977666 (156h18m3.771245316s) [noise=true , address=64.49.13.169:4714 , piece_id=VOVMCNYU7UVDMXN4QVLRQM5OOW6CBEKO2IFHFPMWEKCFNRFK2AVA, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm): protocol: read tcp 5.161.89.252:24186->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 15 05:44:23.040119473 (156h18m3.771387133s) [noise=true , address=156.146.43.225:4440 , piece_id=22Q2FKRAXNHKSOUEZV5CQLKYGYNZSWC2SQNEEFE2OM37II7JX45Q, seg_pos={0 0}, upload=162649]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eqkG9kQB7wFYRNUY3P77MLwBrwXKuEXvJku3B63FZn6rK5y5d): protocol: context canceled; context canceled
Aug 15 05:44:23.040146273 (156h18m3.771413923s) [seg_pos={0 0}, upload=162649]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 15 05:44:23.040241311 (156h18m3.771508962s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 15 05:44:23.040250929 (156h18m3.77151862s) [upload=162649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 15 05:44:23.148699216 (156h18m3.879966877s) [upload=162649]: Done uploading segments...
Aug 15 05:44:23.148718513 (156h18m3.879986163s) [upload=162649]: Upload finished. err: <nil>
Duration: 639.079641ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
168.961476ms
FAIL (2025-08-15 06:10:58 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 168.987104ms
core
uplink
us1
|