netty or socket send message too fast result in losing data packets?
1
I use netty to send files data(master) to other two slaves(All linux OS), send files like this:
The master send 8 files(size:5~200m or so) use Netty NioServerSocketChannel :
if (file.exists() && file.isFile()) {
try {
int count = 0;
int cacheLen = 40960;
if (file.length() % cacheLen == 0) {
count = (int) (file.length() / cacheLen);
} else {
count = (int) (file.length() / cacheLen) + 1;
}
FileInputStream in = new FileInputStream(file);
byte buffer = new byte[cacheLen];
int readed;
int index = 0;
while ((readed = in.read(buffer)) != -1) {
byte tmp = new byte[readed];
System.arraycopy(buffer,0,tmp,0,readed);
FilePack filePack = new FilePack(file.getName(), index, count, tmp);
sendMessage(channel, filePack);
index++;
}
log.info("master to：{}，file：{}，total :{}，sended: {}" ,channel.remoteAddress().toString(),file.getName(),count,index);
} catch (Throwable e) {
log.error("send file error", e);
sendErrInfo(channel,file);
}
}
send method like this:
public SyncGetFuture send(Channel channel, Object message, Long sessionId) throws NetException {
if(!channel.isActive()) {
this.getChannelHub().removeChannelIfDisconnected(channel);
throw new NetException("Failed to send message " + message + " to " + channel.remoteAddress() + ", cause: channel is not active.");
} else if(!channel.isWritable()) {
throw new NetException("Failed to send message " + message + " to " + channel.remoteAddress() + ", cause: channel is not writable.");
} else {
//new an packet
Packet packet = PacketCreator.create(channel.getVersion(), PacketType.BUSINESS, sessionId, message);
try {
nettyChannel.writeAndFlush(packet);
return sessionId != null?this.getSyncFutureManager().newFuture(sessionId):null;
} catch (Throwable var7) {
throw new NetException("Failed to send message " + message + " to " + channel.remoteAddress() + ", cause: " + var7.getMessage(), var7);
}
}
}
netty server:
this.bossGroup = new NioEventLoopGroup(this.configuration.getBossThreads());
this.workerGroup = new NioEventLoopGroup(this.configuration.getWorkerThreads());
final NettyServerHandler4 nettyServerHandler = new NettyServerHandler4(this.configuration, this, (ChannelHub4)this.getChannelHub());
ServerBootstrap b = new ServerBootstrap();
((ServerBootstrap)((ServerBootstrap)b.group(this.bossGroup, this.workerGroup).channel(NioServerSocketChannel.class)).handler(new LoggingHandler(LogLevel.INFO))).childHandler(new ChannelInitializer() {
public void initChannel(SocketChannel ch) throws IOException {
SocketConfig config = SocketServer4.this.configuration.getSocketConfig();
if(config.getWriteBufferHighWaterMark() != null) {
ch.config().setOption(ChannelOption.WRITE_BUFFER_HIGH_WATER_MARK, Integer.valueOf(config.getWriteBufferHighWaterMark().intValue()));
}
if(config.getWriteBufferLowWaterMark() != null) {
ch.config().setOption(ChannelOption.WRITE_BUFFER_LOW_WATER_MARK, Integer.valueOf(config.getWriteBufferLowWaterMark().intValue()));
}
ch.pipeline().addLast("messageDecoder", new PacketDecoder(1048576));
ch.pipeline().addLast("messageEncoder", new PacketEncoder());
ch.pipeline().addLast("idleStateHandler", new IdleStateHandler(0, 0, SocketServer4.this.getConfiguration().getAllIdleTimeSeconds() * 2));
ch.pipeline().addLast("nettyServerHandler", nettyServerHandler);
}
});
netty client:
final NettyClientHandler4 stop = new NettyClientHandler4(this);
Bootstrap i$ = new Bootstrap();
((Bootstrap)((Bootstrap)((Bootstrap)i$.group(this.group)).channel(NioSocketChannel.class)).option(ChannelOption.TCP_NODELAY, Boolean.valueOf(true))).handler(new ChannelInitializer() {
public void initChannel(SocketChannel ch) throws Exception {
SocketConfig config = SocketClient4.this.getConfiguration().getSocketConfig();
if(config.getWriteBufferHighWaterMark() != null) {
ch.config().setOption(ChannelOption.WRITE_BUFFER_HIGH_WATER_MARK, Integer.valueOf(config.getWriteBufferHighWaterMark().intValue()));
}
if(config.getWriteBufferLowWaterMark() != null) {
ch.config().setOption(ChannelOption.WRITE_BUFFER_LOW_WATER_MARK, Integer.valueOf(config.getWriteBufferLowWaterMark().intValue()));
}
ch.pipeline().addLast("messageDecoder", new PacketDecoder(2147483647));
ch.pipeline().addLast("messageEncoder", new PacketEncoder());
ch.pipeline().addLast("idleStateHandler", new IdleStateHandler(0, 0, SocketClient4.this.getConfiguration().getAllIdleTimeSeconds()));
ch.pipeline().addLast("heartBeatHandler", new HeartBeatReqHandler());
ch.pipeline().addLast("nettyClientHandler", stop);
}
});
The last two file or three file always lose data at the slave when the master send all the file at the same time ;
but they can be downloaded successfully when the slaves try to get these files specially the next time;
The log show data lose,file7 and file8 get the last pack suddenly:
16:**:**.*** logback [nioEventLoopGroup-2-1] INFO c.j.c.f.c.l.c.PacketReceiveListener - get file packets,fileName：file7，total：6643，cout:0
.......(downloading 1,2,3,4...5487 correctly)
16:31:18.080 logback [nioEventLoopGroup-2-1] INFO c.j.c.f.c.l.c.PacketReceiveListener - get file packets,fileName：file7，total：6643，cout:5488
16:31:18.097 logback [nioEventLoopGroup-2-1] INFO c.j.c.f.c.l.c.PacketReceiveListener - get file packets,fileName：file7，total：6643，cout:6642
16:31:18.100 logback [nioEventLoopGroup-2-1] INFO c.j.c.f.c.l.c.PacketReceiveListener - get file packets,fileName：file8，total：98，cout:97
no matter master or slave , the logs didn't throw any net exception;
when I and sleep in the while in file send , It works OK:
while ((readed = in.read(buffer)) != -1) {
byte tmp = new byte[readed];
System.arraycopy(buffer,0,tmp,0,readed);
FilePack filePack = new FilePack(file.getName(), index, count, tmp);
sendMessage(channel, filePack);
Thread.sleep(10);
index++;
}
linux sockets socket.io netty netty-socketio
asked Nov 12 '18 at 9:52
Fanl
8221712
add a comment |
1
I use netty to send files data(master) to other two slaves(All linux OS), send files like this:
The master send 8 files(size:5~200m or so) use Netty NioServerSocketChannel :
if (file.exists() && file.isFile()) {
try {
int count = 0;
int cacheLen = 40960;
if (file.length() % cacheLen == 0) {
count = (int) (file.length() / cacheLen);
} else {
count = (int) (file.length() / cacheLen) + 1;
}
FileInputStream in = new FileInputStream(file);
byte buffer = new byte[cacheLen];
int readed;
int index = 0;
while ((readed = in.read(buffer)) != -1) {
byte tmp = new byte[readed];
System.arraycopy(buffer,0,tmp,0,readed);
FilePack filePack = new FilePack(file.getName(), index, count, tmp);
sendMessage(channel, filePack);
index++;
}
log.info("master to：{}，file：{}，total :{}，sended: {}" ,channel.remoteAddress().toString(),file.getName(),count,index);
} catch (Throwable e) {
log.error("send file error", e);
sendErrInfo(channel,file);
}
}
send method like this:
public SyncGetFuture send(Channel channel, Object message, Long sessionId) throws NetException {
if(!channel.isActive()) {
this.getChannelHub().removeChannelIfDisconnected(channel);
throw new NetException("Failed to send message " + message + " to " + channel.remoteAddress() + ", cause: channel is not active.");
} else if(!channel.isWritable()) {
throw new NetException("Failed to send message " + message + " to " + channel.remoteAddress() + ", cause: channel is not writable.");
} else {
//new an packet
Packet packet = PacketCreator.create(channel.getVersion(), PacketType.BUSINESS, sessionId, message);
try {
nettyChannel.writeAndFlush(packet);
return sessionId != null?this.getSyncFutureManager().newFuture(sessionId):null;
} catch (Throwable var7) {
throw new NetException("Failed to send message " + message + " to " + channel.remoteAddress() + ", cause: " + var7.getMessage(), var7);
}
}
}
netty server:
this.bossGroup = new NioEventLoopGroup(this.configuration.getBossThreads());
this.workerGroup = new NioEventLoopGroup(this.configuration.getWorkerThreads());
final NettyServerHandler4 nettyServerHandler = new NettyServerHandler4(this.configuration, this, (ChannelHub4)this.getChannelHub());
ServerBootstrap b = new ServerBootstrap();
((ServerBootstrap)((ServerBootstrap)b.group(this.bossGroup, this.workerGroup).channel(NioServerSocketChannel.class)).handler(new LoggingHandler(LogLevel.INFO))).childHandler(new ChannelInitializer() {
public void initChannel(SocketChannel ch) throws IOException {
SocketConfig config = SocketServer4.this.configuration.getSocketConfig();
if(config.getWriteBufferHighWaterMark() != null) {
ch.config().setOption(ChannelOption.WRITE_BUFFER_HIGH_WATER_MARK, Integer.valueOf(config.getWriteBufferHighWaterMark().intValue()));
}
if(config.getWriteBufferLowWaterMark() != null) {
ch.config().setOption(ChannelOption.WRITE_BUFFER_LOW_WATER_MARK, Integer.valueOf(config.getWriteBufferLowWaterMark().intValue()));
}
ch.pipeline().addLast("messageDecoder", new PacketDecoder(1048576));
ch.pipeline().addLast("messageEncoder", new PacketEncoder());
ch.pipeline().addLast("idleStateHandler", new IdleStateHandler(0, 0, SocketServer4.this.getConfiguration().getAllIdleTimeSeconds() * 2));
ch.pipeline().addLast("nettyServerHandler", nettyServerHandler);
}
});
netty client:
final NettyClientHandler4 stop = new NettyClientHandler4(this);
Bootstrap i$ = new Bootstrap();
((Bootstrap)((Bootstrap)((Bootstrap)i$.group(this.group)).channel(NioSocketChannel.class)).option(ChannelOption.TCP_NODELAY, Boolean.valueOf(true))).handler(new ChannelInitializer() {
public void initChannel(SocketChannel ch) throws Exception {
SocketConfig config = SocketClient4.this.getConfiguration().getSocketConfig();
if(config.getWriteBufferHighWaterMark() != null) {
ch.config().setOption(ChannelOption.WRITE_BUFFER_HIGH_WATER_MARK, Integer.valueOf(config.getWriteBufferHighWaterMark().intValue()));
}
if(config.getWriteBufferLowWaterMark() != null) {
ch.config().setOption(ChannelOption.WRITE_BUFFER_LOW_WATER_MARK, Integer.valueOf(config.getWriteBufferLowWaterMark().intValue()));
}
ch.pipeline().addLast("messageDecoder", new PacketDecoder(2147483647));
ch.pipeline().addLast("messageEncoder", new PacketEncoder());
ch.pipeline().addLast("idleStateHandler", new IdleStateHandler(0, 0, SocketClient4.this.getConfiguration().getAllIdleTimeSeconds()));
ch.pipeline().addLast("heartBeatHandler", new HeartBeatReqHandler());
ch.pipeline().addLast("nettyClientHandler", stop);
}
});
The last two file or three file always lose data at the slave when the master send all the file at the same time ;
but they can be downloaded successfully when the slaves try to get these files specially the next time;
The log show data lose,file7 and file8 get the last pack suddenly:
16:**:**.*** logback [nioEventLoopGroup-2-1] INFO c.j.c.f.c.l.c.PacketReceiveListener - get file packets,fileName：file7，total：6643，cout:0
.......(downloading 1,2,3,4...5487 correctly)
16:31:18.080 logback [nioEventLoopGroup-2-1] INFO c.j.c.f.c.l.c.PacketReceiveListener - get file packets,fileName：file7，total：6643，cout:5488
16:31:18.097 logback [nioEventLoopGroup-2-1] INFO c.j.c.f.c.l.c.PacketReceiveListener - get file packets,fileName：file7，total：6643，cout:6642
16:31:18.100 logback [nioEventLoopGroup-2-1] INFO c.j.c.f.c.l.c.PacketReceiveListener - get file packets,fileName：file8，total：98，cout:97
no matter master or slave , the logs didn't throw any net exception;
when I and sleep in the while in file send , It works OK:
while ((readed = in.read(buffer)) != -1) {
byte tmp = new byte[readed];
System.arraycopy(buffer,0,tmp,0,readed);
FilePack filePack = new FilePack(file.getName(), index, count, tmp);
sendMessage(channel, filePack);
Thread.sleep(10);
index++;
}
linux sockets socket.io netty netty-socketio
asked Nov 12 '18 at 9:52
Fanl
8221712
1
Make sure to check the return result ofwriteAndFlush, it can fail
– Ferrybig
Nov 13 '18 at 12:54
Thanks a lot. The method nettyChannel.writeAndFlush(packet) must add listener to check the cause and then throw the Exception .... The exceptionis : io.netty.util.internal.OutOfDirectMemoryError
– Fanl
Nov 15 '18 at 11:10
Possible duplicate of Netty doesn't write
– Ferrybig
Nov 15 '18 at 11:18
add a comment |
1
1
1
I use netty to send files data(master) to other two slaves(All linux OS), send files like this:
The master send 8 files(size:5~200m or so) use Netty NioServerSocketChannel :
if (file.exists() && file.isFile()) {
try {
int count = 0;
int cacheLen = 40960;
if (file.length() % cacheLen == 0) {
count = (int) (file.length() / cacheLen);
} else {
count = (int) (file.length() / cacheLen) + 1;
}
FileInputStream in = new FileInputStream(file);
byte buffer = new byte[cacheLen];
int readed;
int index = 0;
while ((readed = in.read(buffer)) != -1) {
byte tmp = new byte[readed];
System.arraycopy(buffer,0,tmp,0,readed);
FilePack filePack = new FilePack(file.getName(), index, count, tmp);
sendMessage(channel, filePack);
index++;
}
log.info("master to：{}，file：{}，total :{}，sended: {}" ,channel.remoteAddress().toString(),file.getName(),count,index);
} catch (Throwable e) {
log.error("send file error", e);
sendErrInfo(channel,file);
}
}
send method like this:
public SyncGetFuture send(Channel channel, Object message, Long sessionId) throws NetException {
if(!channel.isActive()) {
this.getChannelHub().removeChannelIfDisconnected(channel);
throw new NetException("Failed to send message " + message + " to " + channel.remoteAddress() + ", cause: channel is not active.");
} else if(!channel.isWritable()) {
throw new NetException("Failed to send message " + message + " to " + channel.remoteAddress() + ", cause: channel is not writable.");
} else {
//new an packet
Packet packet = PacketCreator.create(channel.getVersion(), PacketType.BUSINESS, sessionId, message);
try {
nettyChannel.writeAndFlush(packet);
return sessionId != null?this.getSyncFutureManager().newFuture(sessionId):null;
} catch (Throwable var7) {
throw new NetException("Failed to send message " + message + " to " + channel.remoteAddress() + ", cause: " + var7.getMessage(), var7);
}
}
}
netty server:
this.bossGroup = new NioEventLoopGroup(this.configuration.getBossThreads());
this.workerGroup = new NioEventLoopGroup(this.configuration.getWorkerThreads());
final NettyServerHandler4 nettyServerHandler = new NettyServerHandler4(this.configuration, this, (ChannelHub4)this.getChannelHub());
ServerBootstrap b = new ServerBootstrap();
((ServerBootstrap)((ServerBootstrap)b.group(this.bossGroup, this.workerGroup).channel(NioServerSocketChannel.class)).handler(new LoggingHandler(LogLevel.INFO))).childHandler(new ChannelInitializer() {
public void initChannel(SocketChannel ch) throws IOException {
SocketConfig config = SocketServer4.this.configuration.getSocketConfig();
if(config.getWriteBufferHighWaterMark() != null) {
ch.config().setOption(ChannelOption.WRITE_BUFFER_HIGH_WATER_MARK, Integer.valueOf(config.getWriteBufferHighWaterMark().intValue()));
}
if(config.getWriteBufferLowWaterMark() != null) {
ch.config().setOption(ChannelOption.WRITE_BUFFER_LOW_WATER_MARK, Integer.valueOf(config.getWriteBufferLowWaterMark().intValue()));
}
ch.pipeline().addLast("messageDecoder", new PacketDecoder(1048576));
ch.pipeline().addLast("messageEncoder", new PacketEncoder());
ch.pipeline().addLast("idleStateHandler", new IdleStateHandler(0, 0, SocketServer4.this.getConfiguration().getAllIdleTimeSeconds() * 2));
ch.pipeline().addLast("nettyServerHandler", nettyServerHandler);
}
});
netty client:
final NettyClientHandler4 stop = new NettyClientHandler4(this);
Bootstrap i$ = new Bootstrap();
((Bootstrap)((Bootstrap)((Bootstrap)i$.group(this.group)).channel(NioSocketChannel.class)).option(ChannelOption.TCP_NODELAY, Boolean.valueOf(true))).handler(new ChannelInitializer() {
public void initChannel(SocketChannel ch) throws Exception {
SocketConfig config = SocketClient4.this.getConfiguration().getSocketConfig();
if(config.getWriteBufferHighWaterMark() != null) {
ch.config().setOption(ChannelOption.WRITE_BUFFER_HIGH_WATER_MARK, Integer.valueOf(config.getWriteBufferHighWaterMark().intValue()));
}
if(config.getWriteBufferLowWaterMark() != null) {
ch.config().setOption(ChannelOption.WRITE_BUFFER_LOW_WATER_MARK, Integer.valueOf(config.getWriteBufferLowWaterMark().intValue()));
}
ch.pipeline().addLast("messageDecoder", new PacketDecoder(2147483647));
ch.pipeline().addLast("messageEncoder", new PacketEncoder());
ch.pipeline().addLast("idleStateHandler", new IdleStateHandler(0, 0, SocketClient4.this.getConfiguration().getAllIdleTimeSeconds()));
ch.pipeline().addLast("heartBeatHandler", new HeartBeatReqHandler());
ch.pipeline().addLast("nettyClientHandler", stop);
}
});
The last two file or three file always lose data at the slave when the master send all the file at the same time ;
but they can be downloaded successfully when the slaves try to get these files specially the next time;
The log show data lose,file7 and file8 get the last pack suddenly:
16:**:**.*** logback [nioEventLoopGroup-2-1] INFO c.j.c.f.c.l.c.PacketReceiveListener - get file packets,fileName：file7，total：6643，cout:0
.......(downloading 1,2,3,4...5487 correctly)
16:31:18.080 logback [nioEventLoopGroup-2-1] INFO c.j.c.f.c.l.c.PacketReceiveListener - get file packets,fileName：file7，total：6643，cout:5488
16:31:18.097 logback [nioEventLoopGroup-2-1] INFO c.j.c.f.c.l.c.PacketReceiveListener - get file packets,fileName：file7，total：6643，cout:6642
16:31:18.100 logback [nioEventLoopGroup-2-1] INFO c.j.c.f.c.l.c.PacketReceiveListener - get file packets,fileName：file8，total：98，cout:97
no matter master or slave , the logs didn't throw any net exception;
when I and sleep in the while in file send , It works OK:
while ((readed = in.read(buffer)) != -1) {
byte tmp = new byte[readed];
System.arraycopy(buffer,0,tmp,0,readed);
FilePack filePack = new FilePack(file.getName(), index, count, tmp);
sendMessage(channel, filePack);
Thread.sleep(10);
index++;
}
linux sockets socket.io netty netty-socketio
asked Nov 12 '18 at 9:52
Fanl
8221712
I use netty to send files data(master) to other two slaves(All linux OS), send files like this:
The master send 8 files(size:5~200m or so) use Netty NioServerSocketChannel :
if (file.exists() && file.isFile()) {
try {
int count = 0;
int cacheLen = 40960;
if (file.length() % cacheLen == 0) {
count = (int) (file.length() / cacheLen);
} else {
count = (int) (file.length() / cacheLen) + 1;
}
FileInputStream in = new FileInputStream(file);
byte buffer = new byte[cacheLen];
int readed;
int index = 0;
while ((readed = in.read(buffer)) != -1) {
byte tmp = new byte[readed];
System.arraycopy(buffer,0,tmp,0,readed);
FilePack filePack = new FilePack(file.getName(), index, count, tmp);
sendMessage(channel, filePack);
index++;
}
log.info("master to：{}，file：{}，total :{}，sended: {}" ,channel.remoteAddress().toString(),file.getName(),count,index);
} catch (Throwable e) {
log.error("send file error", e);
sendErrInfo(channel,file);
}
}
send method like this:
public SyncGetFuture send(Channel channel, Object message, Long sessionId) throws NetException {
if(!channel.isActive()) {
this.getChannelHub().removeChannelIfDisconnected(channel);
throw new NetException("Failed to send message " + message + " to " + channel.remoteAddress() + ", cause: channel is not active.");
} else if(!channel.isWritable()) {
throw new NetException("Failed to send message " + message + " to " + channel.remoteAddress() + ", cause: channel is not writable.");
} else {
//new an packet
Packet packet = PacketCreator.create(channel.getVersion(), PacketType.BUSINESS, sessionId, message);
try {
nettyChannel.writeAndFlush(packet);
return sessionId != null?this.getSyncFutureManager().newFuture(sessionId):null;
} catch (Throwable var7) {
throw new NetException("Failed to send message " + message + " to " + channel.remoteAddress() + ", cause: " + var7.getMessage(), var7);
}
}
}
netty server:
this.bossGroup = new NioEventLoopGroup(this.configuration.getBossThreads());
this.workerGroup = new NioEventLoopGroup(this.configuration.getWorkerThreads());
final NettyServerHandler4 nettyServerHandler = new NettyServerHandler4(this.configuration, this, (ChannelHub4)this.getChannelHub());
ServerBootstrap b = new ServerBootstrap();
((ServerBootstrap)((ServerBootstrap)b.group(this.bossGroup, this.workerGroup).channel(NioServerSocketChannel.class)).handler(new LoggingHandler(LogLevel.INFO))).childHandler(new ChannelInitializer() {
public void initChannel(SocketChannel ch) throws IOException {
SocketConfig config = SocketServer4.this.configuration.getSocketConfig();
if(config.getWriteBufferHighWaterMark() != null) {
ch.config().setOption(ChannelOption.WRITE_BUFFER_HIGH_WATER_MARK, Integer.valueOf(config.getWriteBufferHighWaterMark().intValue()));
}
if(config.getWriteBufferLowWaterMark() != null) {
ch.config().setOption(ChannelOption.WRITE_BUFFER_LOW_WATER_MARK, Integer.valueOf(config.getWriteBufferLowWaterMark().intValue()));
}
ch.pipeline().addLast("messageDecoder", new PacketDecoder(1048576));
ch.pipeline().addLast("messageEncoder", new PacketEncoder());
ch.pipeline().addLast("idleStateHandler", new IdleStateHandler(0, 0, SocketServer4.this.getConfiguration().getAllIdleTimeSeconds() * 2));
ch.pipeline().addLast("nettyServerHandler", nettyServerHandler);
}
});
netty client:
final NettyClientHandler4 stop = new NettyClientHandler4(this);
Bootstrap i$ = new Bootstrap();
((Bootstrap)((Bootstrap)((Bootstrap)i$.group(this.group)).channel(NioSocketChannel.class)).option(ChannelOption.TCP_NODELAY, Boolean.valueOf(true))).handler(new ChannelInitializer() {
public void initChannel(SocketChannel ch) throws Exception {
SocketConfig config = SocketClient4.this.getConfiguration().getSocketConfig();
if(config.getWriteBufferHighWaterMark() != null) {
ch.config().setOption(ChannelOption.WRITE_BUFFER_HIGH_WATER_MARK, Integer.valueOf(config.getWriteBufferHighWaterMark().intValue()));
}
if(config.getWriteBufferLowWaterMark() != null) {
ch.config().setOption(ChannelOption.WRITE_BUFFER_LOW_WATER_MARK, Integer.valueOf(config.getWriteBufferLowWaterMark().intValue()));
}
ch.pipeline().addLast("messageDecoder", new PacketDecoder(2147483647));
ch.pipeline().addLast("messageEncoder", new PacketEncoder());
ch.pipeline().addLast("idleStateHandler", new IdleStateHandler(0, 0, SocketClient4.this.getConfiguration().getAllIdleTimeSeconds()));
ch.pipeline().addLast("heartBeatHandler", new HeartBeatReqHandler());
ch.pipeline().addLast("nettyClientHandler", stop);
}
});
The last two file or three file always lose data at the slave when the master send all the file at the same time ;
but they can be downloaded successfully when the slaves try to get these files specially the next time;
The log show data lose,file7 and file8 get the last pack suddenly:
16:**:**.*** logback [nioEventLoopGroup-2-1] INFO c.j.c.f.c.l.c.PacketReceiveListener - get file packets,fileName：file7，total：6643，cout:0
.......(downloading 1,2,3,4...5487 correctly)
16:31:18.080 logback [nioEventLoopGroup-2-1] INFO c.j.c.f.c.l.c.PacketReceiveListener - get file packets,fileName：file7，total：6643，cout:5488
16:31:18.097 logback [nioEventLoopGroup-2-1] INFO c.j.c.f.c.l.c.PacketReceiveListener - get file packets,fileName：file7，total：6643，cout:6642
16:31:18.100 logback [nioEventLoopGroup-2-1] INFO c.j.c.f.c.l.c.PacketReceiveListener - get file packets,fileName：file8，total：98，cout:97
no matter master or slave , the logs didn't throw any net exception;
when I and sleep in the while in file send , It works OK:
while ((readed = in.read(buffer)) != -1) {
byte tmp = new byte[readed];
System.arraycopy(buffer,0,tmp,0,readed);
FilePack filePack = new FilePack(file.getName(), index, count, tmp);
sendMessage(channel, filePack);
Thread.sleep(10);
index++;
}
linux sockets socket.io netty netty-socketio
linux sockets socket.io netty netty-socketio
asked Nov 12 '18 at 9:52
Fanl
8221712
asked Nov 12 '18 at 9:52
Fanl
8221712
asked Nov 12 '18 at 9:52
Fanl
8221712
asked Nov 12 '18 at 9:52
Fanl
8221712
asked Nov 12 '18 at 9:52
Fanl
8221712
8221712
1
Make sure to check the return result ofwriteAndFlush, it can fail
– Ferrybig
Nov 13 '18 at 12:54
Thanks a lot. The method nettyChannel.writeAndFlush(packet) must add listener to check the cause and then throw the Exception .... The exceptionis : io.netty.util.internal.OutOfDirectMemoryError
– Fanl
Nov 15 '18 at 11:10
Possible duplicate of Netty doesn't write
– Ferrybig
Nov 15 '18 at 11:18
add a comment |
1
Make sure to check the return result ofwriteAndFlush, it can fail
– Ferrybig
Nov 13 '18 at 12:54
Thanks a lot. The method nettyChannel.writeAndFlush(packet) must add listener to check the cause and then throw the Exception .... The exceptionis : io.netty.util.internal.OutOfDirectMemoryError
– Fanl
Nov 15 '18 at 11:10
Possible duplicate of Netty doesn't write
– Ferrybig
Nov 15 '18 at 11:18
1
1
Make sure to check the return result of
writeAndFlush, it can fail– Ferrybig
Nov 13 '18 at 12:54
Make sure to check the return result of
writeAndFlush, it can fail– Ferrybig
Nov 13 '18 at 12:54
Thanks a lot. The method nettyChannel.writeAndFlush(packet) must add listener to check the cause and then throw the Exception .... The exceptionis : io.netty.util.internal.OutOfDirectMemoryError
– Fanl
Nov 15 '18 at 11:10
Thanks a lot. The method nettyChannel.writeAndFlush(packet) must add listener to check the cause and then throw the Exception .... The exceptionis : io.netty.util.internal.OutOfDirectMemoryError
– Fanl
Nov 15 '18 at 11:10
Possible duplicate of Netty doesn't write
– Ferrybig
Nov 15 '18 at 11:18
Possible duplicate of Netty doesn't write
– Ferrybig
Nov 15 '18 at 11:18
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53259596%2fnetty-or-socket-send-message-too-fast-result-in-losing-data-packets%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53259596%2fnetty-or-socket-send-message-too-fast-result-in-losing-data-packets%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
Make sure to check the return result of
writeAndFlush, it can fail– Ferrybig
Nov 13 '18 at 12:54
Thanks a lot. The method nettyChannel.writeAndFlush(packet) must add listener to check the cause and then throw the Exception .... The exceptionis : io.netty.util.internal.OutOfDirectMemoryError
– Fanl
Nov 15 '18 at 11:10
Possible duplicate of Netty doesn't write
– Ferrybig
Nov 15 '18 at 11:18