iptables restart带来的隐性问题

2016年6月2日 没有评论

高并发服务器,忽然性能变差,几经周折,某大牛发现一些内核参数因iptables的调整被自动重置了。
比如:/proc/sys/net/netfilter/nf_conntrack_max 被复位成65535了

解决办法:
修改/etc/sysconfig/iptables-config中的IPTABLES_MODULES_UNLOAD为no即可解决:

# Unload modules on restart and stop
#   Value: yes|no,  default: yes
# This option has to be 'yes' to get to a sane state for a firewall
# restart or stop. Only set to 'no' if there are problems unloading netfilter
# modules.
IPTABLES_MODULES_UNLOAD="no"

iptables的调整很常见,我从来没意识到会引起netfilter模块参数的重置。估计会有很多小白像我一样~

黑科技:Kafka之绑定Host问题

2016年5月20日 没有评论

之前搞定了HBase的绑定Host问题,今天在Kafka上用的时候,发现还是有问题,于是又换了一种更彻底的实现,可以直接取代之前的搞法。
本次问题表现:

Caused by: java.net.UnknownHostException: hostxxx: hostxxx
        at java.net.InetAddress.getLocalHost(InetAddress.java:1496) ~[na:1.7.0_101]
        at kafka.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:119) ~[kafka_2.10-0.8.2.2.jar!/:na]
        at kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:66) ~[kafka_2.10-0.8.2.2.jar!/:na]
        at kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:69) ~[kafka_2.10-0.8.2.2.jar!/:na]
        at kafka.consumer.Consumer$.createJavaConsumerConnector(ConsumerConnector.scala:105) ~[kafka_2.10-0.8.2.2.jar!/:na]
        at kafka.consumer.Consumer.createJavaConsumerConnector(ConsumerConnector.scala) ~[kafka_2.10-0.8.2.2.jar!/:na]
        ... 53 common frames omitted
Caused by: java.net.UnknownHostException: hostxxx
        at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method) ~[na:1.7.0_101]
        at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:922) ~[na:1.7.0_101]
        at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1316) ~[na:1.7.0_101]
        at java.net.InetAddress.getLocalHost(InetAddress.java:1492) ~[na:1.7.0_101]
        ... 69 common frames omitted

更多内容…

分类: 工作 标签: , , ,

黑科技:HBase之绑定Host问题

2016年4月28日 没有评论

访问HBase时,必须绑定host,否则那种hostname没办法解析。
作为致力于降低生产环境运维难度的选手,挖到一个办法来解决这个问题。

hbase的hostname判断位于org.apache.hadoop.hbase.ipc.RpcClientImpl的第299行

    Connection(ConnectionId remoteId, final Codec codec, final CompressionCodec compressor)
    throws IOException {
      if (remoteId.getAddress().isUnresolved()) {
        throw new UnknownHostException("unknown host: " + remoteId.getAddress().getHostName());
      }

其中remoteId.getAddress()和定义如下

  public InetSocketAddress getAddress() {
    return address;
  }

InetSocketAddress的.isUnresolved()如下

    private final transient InetSocketAddressHolder holder;
...
    public final boolean isUnresolved() {
        return holder.isUnresolved();
    }

InetSocketAddressHolder的isUnresolved实现

        private boolean isUnresolved() {
            return addr == null;
        }

也就是说:只要InetSocketAddressHolder中的addr值不为null(为正常值),即解析成功。
更多内容…

java语言中常用时间戳的值域

2016年4月20日 没有评论

经常使用下面这两个时间值当唯一标识,却总记不住它们的时间范围,记录之。

System.currentTimeMillis();
System.nanoTime();

其中currentTimeMillis的时间范围可以简单代码测试得到。
先看描述

 long java.lang.System.currentTimeMillis()


Returns the current time in milliseconds. Note that while the unit of time of the return value is a millisecond, the granularity of the value depends on the underlying operating system and may be larger. For example, many operating systems measure time in units of tens of milliseconds. 

See the description of the class Date for a discussion of slight discrepancies that may arise between "computer time" and coordinated universal time (UTC).

Returns:
the difference, measured in milliseconds, between the current time and midnight, January 1, 1970 UTC.
See Also:
java.util.Date

再看代码

Calendar c = Calendar.getInstance();
c.setTimeInMillis(Long.MAX_VALUE);
System.out.println(c.get(Calendar.YEAR));

输入结果为:292278994,加上几千分符292,278,994 = 0.29billion
单位是年,将三亿年,咱们肯定都活不到那时候了~ 人类那时候还在不在也不好说了~ 应该够用了~~

再看nanoTime的描述

 long java.lang.System.nanoTime()


Returns the current value of the running Java Virtual Machine's high-resolution time source, in nanoseconds. 

This method can only be used to measure elapsed time and is not related to any other notion of system or wall-clock time. The value returned represents nanoseconds since some fixed but arbitrary origin time (perhaps in the future, so values may be negative). The same origin is used by all invocations of this method in an instance of a Java virtual machine; other virtual machine instances are likely to use a different origin. 

This method provides nanosecond precision, but not necessarily nanosecond resolution (that is, how frequently the value changes) - no guarantees are made except that the resolution is at least as good as that of currentTimeMillis(). 

Differences in successive calls that span greater than approximately 292 years (2^63 nanoseconds) will not correctly compute elapsed time due to numerical overflow. 

The values returned by this method become meaningful only when the difference between two such values, obtained within the same instance of a Java virtual machine, is computed. 

For example, to measure how long some code takes to execute: 

 long startTime = System.nanoTime();
 // ... the code being measured ...
 long estimatedTime = System.nanoTime() - startTime;
To compare two nanoTime values 

 long t0 = System.nanoTime();
 ...
 long t1 = System.nanoTime();
one should use t1 - t0 < 0, not t1 < t0, because of the possibility of numerical overflow.
Returns:
the current value of the running Java Virtual Machine's high-resolution time source, in nanoseconds
Since:
1.5

人家已经说得很清楚,区间为292年。还是算一下能用到哪一年吧。

long nanosToMax = Long.MAX_VALUE - System.nanoTime();// 还剩下的nanos数
long millisToMax = TimeUnit.NANOSECONDS.toMillis(nanosToMax);// 转为剩下的毫秒数
long maxMillis = System.currentTimeMillis() + millisToMax;// 根据当前时间算出的最大nano对应的millis时间
Calendar c = Calendar.getInstance();
c.setTimeInMillis(maxMillis);
System.out.println(c.get(Calendar.YEAR));

运行结果是:2308
2308 – 2016 = 292
这个数字居然跟人家描述里说得一样,总觉得哪不对~ 好吧,还有两百多年可以用,我们也都不会见到那一刻了~

postgresql用于序列号生成器备忘

2016年4月6日 没有评论

第一步,安装:

yum install postgresql-server

第二步,指定并初始化目录

cd /data/
mkdir psqldata
chown postgres:postgres psqldata
su - postgres
initdb /data/psqldata/

第三步,修改基本配置postgresql.conf

vi /data/psqldata/postgresql.conf
#改几个基本配置listen_addresses、port、max_connections

vi /data/psqldata/pg_hba.conf
#在IPv4 local connections:下增加相应IP的授权

更多内容…

Spring with tomcat中文乱码问题

2016年3月11日 没有评论

spring with tomcat,中文乱码,相关版本:

<spring-framework.version>4.0.8.RELEASE</spring-framework.version>
<spring-boot.version>1.1.9.RELEASE</spring-boot.version>

在使用request之前对其设置utf-8编码没效果,debug时发现,其charset为iso-8859-1。

更多内容…

分类: 工作 标签: , ,