大数据环境下任务调度和资源部署研究

文章摘要

大数据环境下任务调度和资源部署研究

Research on Task Scheduling and Resource Deployment in Big Data Environment

投稿时间：2019-09-29 修订日期：2019-09-29

DOI：

中文关键词: 大数据 HDSF 任务调度资源预测

英文关键词: Large Data HDSF Task Scheduling Resource Forecasting

基金项目:福建省教育厅A类课题：逆向工程技术在高精度模具开发中的应用研究（编号JAT170984 ）。

作者	单位	邮编
蔡尊煌^*	福建林业职业技术学院福建南平 353000	353000

摘要点击次数: 84

全文下载次数: 0

中文摘要:

随着计算机技术的不断发展,网络数据正呈现指数倍增长,逐步形成大数据环境。大数据对事务处理和预测带来了很大的便利,因此得到广泛的关注和研究。基于以上背景,文章对于大数据环境下任务调度和资源部署算法进行了研究,首先建立了任务调度和资源部署模型,模型包含HDSF集群、数据节点磁盘输入输出负载、网络负载、磁盘输入输出矩阵描述,然后基于矩阵描述文章设计了任务调度和资源部署的基本流程,并通过任务排程器对流程进行了实现；最后通过实验验证分析,在高负荷、低负荷和随机负荷件下,文章所涉及算法相较于离请求者最近算法、随机算法在资源评价结果方面具有较大优势,也验证了文章所提出算法的合理性和可用性。

英文摘要:

With the continuous development of information technology, network and information technology are constantly applied in all walks of life, network data is showing exponential growth, gradually forming a large data environment. Large data has brought great convenience to transaction processing and prediction, so it has been widely concerned and studied. Based on the above background, this paper studies task scheduling and resource deployment algorithm in large data environment. Firstly, this paper establishes task scheduling and resource deployment model, which includes HDSF cluster, data node disk input and output load, network load, disk input and output matrix description. Then, based on matrix description, this paper designs the basic flow of task scheduling and resource deployment. The process is implemented by task scheduler. At last, the experimental results show that under the conditions of high load, low load and random coincidence, the algorithm in this paper has more advantages than the nearest algorithm and the stochastic algorithm in resource evaluation, and it also verifies the rationality and availability of the algorithm proposed in this paper.

View Fulltext 查看/发表评论下载PDF阅读器

关闭