zookeeper+kafka集群安装之二

作者: 云计算机网 分类: 云计算知识 发布时间: 2016-04-30 17:22

zookeeper+kafka集群安装之二

此为上一篇文章的续篇, kafka安装需要依赖zookeeper, 本文与上一篇文章都是真正分布式安装配置, 可以直接用于生产环境.

zookeeper安装参考:

http://blog.csdn.net/ubuntu64fan/article/details/26678877

首先了解几个kafka中的概念:kafka是一个消息队列服务器,服务称为broker, 消息发送者称为producer, 消息接收者称为consumer;通常我们部署多个broker以提供高可用性的消息服务集群.典型的是3个broker;消息以topic的形式发送到broker,消费者订阅topic,实现按需取用的消费模式;创建topic需要指定replication-factor(复制数目, 通常=broker数目);每个topic可能有多个分区(partition), 每个分区的消息内容不会重复:

假定我们有一个名称为test的topic, 分区数目为2, 当我们发送到这个test具体的消息"msg1:hello beijing"和"msg2:hello shanghai"的时候,我们如何知道消息的发送路径呢(发往哪个分区)?msg1如果被发送到分区test.1,则肯定不会发送到test.2. 数据发送路径选择决策受kafka.producer.Partitioner的影响:interface Partitioner {    int partition(java.lang.Object key, int numPartitions);}

一个伪代码的实现如下:

package org.mymibao.mq.client;import kafka.producer.Partitioner;public class DefaultKafkaPartitioner implements Partitioner {    private final static int FIRST_PARTITION_ID = 1;    public int partition(Object key, int numPartitions) {        return FIRST_PARTITION_ID;    }}

分区API根据相关的键值以及系统中具有的代理分区的数量返回一个分区id。将该id用作索引,在broker_id和partition组成的经过排序的列表中为相应的生产者请求找出一个代理分区。缺省的分区策略是hash(key)%numPartitions。如果key为null,那就进行随机选择。使用partitioner.class这个配置参数可用插入自定义的分区策略.分区文件不会跨越broker,但是多个broker上可以有某个topic的分区副本.

kafka安装配置参考:

1)下载KAFKA

$ wget http://apache.fayea.com/apache-mirror/kafka/0.8.1.1/kafka_2.9.2-0.8.1.1.tgz

安装和配置参考上一篇文章:

http://blog.csdn.net/ubuntu64fan/article/details/26678877

2)配置$KAFKA_HOME/config/server.properties

我们安装3个broker,分别在3个vm上:zk1,zk2,zk3:

zk1:

$ vi /etc/sysconfig/network

NETWORKING=yesHOSTNAME=zk1

$ vi $KAFKA_HOME/config/server.properties

broker.id=0port=9092host.name=zk1advertised.host.name=zk1...num.partitions=2...zookeeper.contact=zk1:2181,zk2:2181,zk3:2181

zk2:

$ vi /etc/sysconfig/network

NETWORKING=yesHOSTNAME=zk2

$ vi $KAFKA_HOME/config/server.properties

broker.id=1port=9092host.name=zk2advertised.host.name=zk2...num.partitions=2...zookeeper.contact=zk1:2181,zk2:2181,zk3:2181 

  • 以前我们只知道在Linux下构建ftp服务器可以使用wuftp,现在看看网上,几乎所有的人都使用proftpd,下面爱站技术频道小编详解安装配置proftpd搭建ftp服务器的教程。

    以前我们只知道在Linux下构建ftp服务器可以使用wuftp,现在看看网上,几乎所有的人都使用proftpd,下面爱站技术频道小编详解安装配置proftpd搭建ftp服务器的教程。

    proftpd全称:Professional FTP daemon,是针对Wu-FTP的弱项而开发的,除了改进的安全性,还具备许多Wu-FTP没有的特点,能以Stand-alone、xinetd模式运行等。ProFTP已经成为继Wu-FTP之后最为流行的FTP服务器软件,越来越多的站点选用它构筑安全高效的FTP站点,ProFTP配置方便,并有MySQL和Quota模块可供选择,利用它们的完美结合可以实现非系统账号的管理和用户磁盘的限制。lt;摘抄百度百科gt;

    本章通过下载源码的方式安装,可以到官网下载最新版本:http://www.proftpd.org/

    1、首先安装lrzsz方便拷贝文件到要搭建服务器的linux路径下

    yum install lrzsz

    2、上传文件解压

    将下载完成的proftpd-1.3.6.tar.gz文件上传到指定路径,使用tar zxvf proftpd-1.3.6.tar.gz解压

    3、由于使用源码安装,所以需要安装gcc编译环境

    yum install gcc gcc-c++ autoconf automake

    3、配置并制定安装和配置文件路径

    ./configure --prefix=/usr/local/proftpd --sysconfdir=/usr/local/proftpd

    4、安装

    makeamp;make install

    5、修改配置文件,限制匿名用户可以上传下载,但是不能删除(本章暂不详细描述权限控制部分内容)

    vi /usr/local/proftpd/proftpd.conf# This is a basic ProFTPD configuration file (rename it to # 'proftpd.conf' for actual use. It establishes a single server# and a single anonymous login. It assumes that you have a user/group# "nobody" and "ftp" for normal operation and anon.ServerName           "Welcome to FTP Server"ServerType           standaloneDefaultServer          on# Port 21 is the standard FTP port.Port              21# Don't use IPv6 support by default.UseIPv6             off# Umask 022 is a good standard umask to prevent new dirs and files# from being group and world writable.Umask              022# To prevent DoS attacks, set the maximum number of child processes# to 30. If you need to allow more than 30 concurrent connections# at once, simply increase this value. Note that this ONLY works# in standalone mode, in inetd mode you should use an inetd server# that allows you to limit maximum number of processes per service# (such as xinetd).MaxInstances          30# Set the user and group under which the server will run.User              nobodyGroup              nobody# To cause every FTP user to be "jailed" (chrooted) into their home# directory, uncomment this line.#DefaultRoot ~# Normally, we want files to be overwriteable.AllowOverwrite     onIdentLookups      offUseReverseDNS      off# Bar use of SITE CHMOD by default AllowAll#p#分页标题#e## A basic anonymous configuration, no upload directories. If you do not# want anonymous users, simply delete this entire  section. User             ftp Group             ftp # We want clients to be able to login with "anonymous" as well as "ftp" UserAlias           anonymous ftp # Limit the maximum number of anonymous logins MaxClients          50 # We want 'welcome.msg' displayed at login, and '.message' displayed # in each newly chdired directory. DisplayLogin         welcome.msg DisplayChdir         .message # Limit WRITE everywhere in the anonymous chroot   DenyAll 

    6、关闭防火墙

    systemctl stop firewalld.servicesystemctl disable firewalld.service

    7、启动服务器

    /usr/local/proftpd/sbin/proftpd

    查看进程是否已经启动

    [root@localhost ~]# ps -ef |grep proftpdnobody  1140   1 0 12:13 ?    00:00:00 proftpd: (accepting connections)root   3182 2270 0 12:50 pts/0  00:00:00 grep --color=auto proftpd 

    8、添加到启动项避免每次重启后都需要手动启动

    vi /etc/rc.d/rc.local#!/bin/bash# THIS FILE IS ADDED FOR COMPATIBILITY PURPOSES## It is highly advisable to create own systemd services or udev rules# to run scripts during boot instead of using this file.## In contrast to previous versions due to parallel execution during boot# this script will NOT be run after all other services.## Please note that you must run 'chmod +x /etc/rc.d/rc.local' to ensure# that this script will be executed during boot.touch /var/lock/subsys/local/usr/local/proftpd/sbin/proftpd

    由于在centos7中/etc/rc.d/rc.local的权限被降低了,所以需要加上可执行的权限:

    chmod +x /etc/rc.d/rc.local

    总结

    以上所述是爱站技术频道小编给大家详解安装配置proftpd搭建ftp服务器的教程,希望对大家有所帮助,如果大家有任何疑问请给我留言。

  • 相关推荐:

  • 详解安装配置proftpd搭建
  • 怎么安装vsftpd
  • 2018常熟跨境电商峰会,
  • 为什么Linux到现在还不能
  • 临夏县教育局2019年临夏
  • 物探领域新突破,曙光斩
  • Windows7iis配置办法window
  • 迅雷下载提示未安装迅雷
  • Mac系统怎么安装驱动精灵
  • photoshop字体安装方法
  • 网站内容禁止违规转载,转载授权联系中国云计算网