服务器 Netty网络编程 - NIO基础

一. NIO 基础

non-blocking io 非阻塞 IO

1. 三大组件

1.1 Channel & Buffer

channel 有一点类似于 stream，它就是读写数据的双向通道，可以从 channel 将数据读入 buffer，也可以将 buffer 的数据写入 channel，而之前的 stream 要么是输入，要么是输出，channel 比 stream 更为底层

#mermaid-svg-AG1HN1Zc14MPH3Wb {font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}#mermaid-svg-AG1HN1Zc14MPH3Wb .error-icon{fill:#552222;}#mermaid-svg-AG1HN1Zc14MPH3Wb .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-AG1HN1Zc14MPH3Wb .edge-thickness-normal{stroke-width:2px;}#mermaid-svg-AG1HN1Zc14MPH3Wb .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-AG1HN1Zc14MPH3Wb .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-AG1HN1Zc14MPH3Wb .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-AG1HN1Zc14MPH3Wb .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-AG1HN1Zc14MPH3Wb .marker{fill:#333333;stroke:#333333;}#mermaid-svg-AG1HN1Zc14MPH3Wb .marker.cross{stroke:#333333;}#mermaid-svg-AG1HN1Zc14MPH3Wb svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-AG1HN1Zc14MPH3Wb .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-AG1HN1Zc14MPH3Wb .cluster-label text{fill:#333;}#mermaid-svg-AG1HN1Zc14MPH3Wb .cluster-label span{color:#333;}#mermaid-svg-AG1HN1Zc14MPH3Wb .label text,#mermaid-svg-AG1HN1Zc14MPH3Wb span{fill:#333;color:#333;}#mermaid-svg-AG1HN1Zc14MPH3Wb .node rect,#mermaid-svg-AG1HN1Zc14MPH3Wb .node circle,#mermaid-svg-AG1HN1Zc14MPH3Wb .node ellipse,#mermaid-svg-AG1HN1Zc14MPH3Wb .node polygon,#mermaid-svg-AG1HN1Zc14MPH3Wb .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-AG1HN1Zc14MPH3Wb .node .label{text-align:center;}#mermaid-svg-AG1HN1Zc14MPH3Wb .node.clickable{cursor:pointer;}#mermaid-svg-AG1HN1Zc14MPH3Wb .arrowheadPath{fill:#333333;}#mermaid-svg-AG1HN1Zc14MPH3Wb .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-AG1HN1Zc14MPH3Wb .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-AG1HN1Zc14MPH3Wb .edgeLabel{background-color:#e8e8e8;text-align:center;}#mermaid-svg-AG1HN1Zc14MPH3Wb .edgeLabel rect{opacity:0.5;background-color:#e8e8e8;fill:#e8e8e8;}#mermaid-svg-AG1HN1Zc14MPH3Wb .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-AG1HN1Zc14MPH3Wb .cluster text{fill:#333;}#mermaid-svg-AG1HN1Zc14MPH3Wb .cluster span{color:#333;}#mermaid-svg-AG1HN1Zc14MPH3Wb div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-AG1HN1Zc14MPH3Wb :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;}

channel

buffer

常见的 Channel 有

FileChannelDatagramChannelSocketChannelServerSocketChannel

buffer 则用来缓冲读写数据，常见的 buffer 有

ByteBuffer

MappedByteBufferDirectByteBufferHeapByteBuffer ShortBufferIntBufferLongBufferFloatBufferDoubleBufferCharBuffer

1.2 Selector

selector 单从字面意思不好理解，需要结合服务器的设计演化来理解它的用途

多线程版设计

#mermaid-svg-fUz3djOTcriB10O8 {font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}#mermaid-svg-fUz3djOTcriB10O8 .error-icon{fill:#552222;}#mermaid-svg-fUz3djOTcriB10O8 .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-fUz3djOTcriB10O8 .edge-thickness-normal{stroke-width:2px;}#mermaid-svg-fUz3djOTcriB10O8 .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-fUz3djOTcriB10O8 .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-fUz3djOTcriB10O8 .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-fUz3djOTcriB10O8 .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-fUz3djOTcriB10O8 .marker{fill:#333333;stroke:#333333;}#mermaid-svg-fUz3djOTcriB10O8 .marker.cross{stroke:#333333;}#mermaid-svg-fUz3djOTcriB10O8 svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-fUz3djOTcriB10O8 .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-fUz3djOTcriB10O8 .cluster-label text{fill:#333;}#mermaid-svg-fUz3djOTcriB10O8 .cluster-label span{color:#333;}#mermaid-svg-fUz3djOTcriB10O8 .label text,#mermaid-svg-fUz3djOTcriB10O8 span{fill:#333;color:#333;}#mermaid-svg-fUz3djOTcriB10O8 .node rect,#mermaid-svg-fUz3djOTcriB10O8 .node circle,#mermaid-svg-fUz3djOTcriB10O8 .node ellipse,#mermaid-svg-fUz3djOTcriB10O8 .node polygon,#mermaid-svg-fUz3djOTcriB10O8 .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-fUz3djOTcriB10O8 .node .label{text-align:center;}#mermaid-svg-fUz3djOTcriB10O8 .node.clickable{cursor:pointer;}#mermaid-svg-fUz3djOTcriB10O8 .arrowheadPath{fill:#333333;}#mermaid-svg-fUz3djOTcriB10O8 .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-fUz3djOTcriB10O8 .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-fUz3djOTcriB10O8 .edgeLabel{background-color:#e8e8e8;text-align:center;}#mermaid-svg-fUz3djOTcriB10O8 .edgeLabel rect{opacity:0.5;background-color:#e8e8e8;fill:#e8e8e8;}#mermaid-svg-fUz3djOTcriB10O8 .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-fUz3djOTcriB10O8 .cluster text{fill:#333;}#mermaid-svg-fUz3djOTcriB10O8 .cluster span{color:#333;}#mermaid-svg-fUz3djOTcriB10O8 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-fUz3djOTcriB10O8 :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;}

多线程版

socket1

thread

socket2

thread

socket3

thread

⚠️ 多线程版缺点

内存占用高线程上下文切换成本高只适合连接数少的场景

线程池版设计

#mermaid-svg-YLieHIj4qyxXyfVs {font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}#mermaid-svg-YLieHIj4qyxXyfVs .error-icon{fill:#552222;}#mermaid-svg-YLieHIj4qyxXyfVs .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-YLieHIj4qyxXyfVs .edge-thickness-normal{stroke-width:2px;}#mermaid-svg-YLieHIj4qyxXyfVs .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-YLieHIj4qyxXyfVs .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-YLieHIj4qyxXyfVs .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-YLieHIj4qyxXyfVs .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-YLieHIj4qyxXyfVs .marker{fill:#333333;stroke:#333333;}#mermaid-svg-YLieHIj4qyxXyfVs .marker.cross{stroke:#333333;}#mermaid-svg-YLieHIj4qyxXyfVs svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-YLieHIj4qyxXyfVs .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-YLieHIj4qyxXyfVs .cluster-label text{fill:#333;}#mermaid-svg-YLieHIj4qyxXyfVs .cluster-label span{color:#333;}#mermaid-svg-YLieHIj4qyxXyfVs .label text,#mermaid-svg-YLieHIj4qyxXyfVs span{fill:#333;color:#333;}#mermaid-svg-YLieHIj4qyxXyfVs .node rect,#mermaid-svg-YLieHIj4qyxXyfVs .node circle,#mermaid-svg-YLieHIj4qyxXyfVs .node ellipse,#mermaid-svg-YLieHIj4qyxXyfVs .node polygon,#mermaid-svg-YLieHIj4qyxXyfVs .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-YLieHIj4qyxXyfVs .node .label{text-align:center;}#mermaid-svg-YLieHIj4qyxXyfVs .node.clickable{cursor:pointer;}#mermaid-svg-YLieHIj4qyxXyfVs .arrowheadPath{fill:#333333;}#mermaid-svg-YLieHIj4qyxXyfVs .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-YLieHIj4qyxXyfVs .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-YLieHIj4qyxXyfVs .edgeLabel{background-color:#e8e8e8;text-align:center;}#mermaid-svg-YLieHIj4qyxXyfVs .edgeLabel rect{opacity:0.5;background-color:#e8e8e8;fill:#e8e8e8;}#mermaid-svg-YLieHIj4qyxXyfVs .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-YLieHIj4qyxXyfVs .cluster text{fill:#333;}#mermaid-svg-YLieHIj4qyxXyfVs .cluster span{color:#333;}#mermaid-svg-YLieHIj4qyxXyfVs div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-YLieHIj4qyxXyfVs :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;}

线程池版

socket1

thread

socket2

thread

socket3

socket4

⚠️ 线程池版缺点

阻塞模式下，线程仅能处理一个 socket 连接仅适合短连接场景

selector 版设计

selector 的作用就是配合一个线程来管理多个 channel，获取这些 channel 上发生的事件，这些 channel 工作在非阻塞模式下，不会让线程吊死在一个 channel 上。适合连接数特别多，但流量低的场景（low traffic）

#mermaid-svg-IjcZ7IERqoxPunah {font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}#mermaid-svg-IjcZ7IERqoxPunah .error-icon{fill:#552222;}#mermaid-svg-IjcZ7IERqoxPunah .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-IjcZ7IERqoxPunah .edge-thickness-normal{stroke-width:2px;}#mermaid-svg-IjcZ7IERqoxPunah .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-IjcZ7IERqoxPunah .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-IjcZ7IERqoxPunah .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-IjcZ7IERqoxPunah .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-IjcZ7IERqoxPunah .marker{fill:#333333;stroke:#333333;}#mermaid-svg-IjcZ7IERqoxPunah .marker.cross{stroke:#333333;}#mermaid-svg-IjcZ7IERqoxPunah svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-IjcZ7IERqoxPunah .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-IjcZ7IERqoxPunah .cluster-label text{fill:#333;}#mermaid-svg-IjcZ7IERqoxPunah .cluster-label span{color:#333;}#mermaid-svg-IjcZ7IERqoxPunah .label text,#mermaid-svg-IjcZ7IERqoxPunah span{fill:#333;color:#333;}#mermaid-svg-IjcZ7IERqoxPunah .node rect,#mermaid-svg-IjcZ7IERqoxPunah .node circle,#mermaid-svg-IjcZ7IERqoxPunah .node ellipse,#mermaid-svg-IjcZ7IERqoxPunah .node polygon,#mermaid-svg-IjcZ7IERqoxPunah .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-IjcZ7IERqoxPunah .node .label{text-align:center;}#mermaid-svg-IjcZ7IERqoxPunah .node.clickable{cursor:pointer;}#mermaid-svg-IjcZ7IERqoxPunah .arrowheadPath{fill:#333333;}#mermaid-svg-IjcZ7IERqoxPunah .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-IjcZ7IERqoxPunah .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-IjcZ7IERqoxPunah .edgeLabel{background-color:#e8e8e8;text-align:center;}#mermaid-svg-IjcZ7IERqoxPunah .edgeLabel rect{opacity:0.5;background-color:#e8e8e8;fill:#e8e8e8;}#mermaid-svg-IjcZ7IERqoxPunah .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-IjcZ7IERqoxPunah .cluster text{fill:#333;}#mermaid-svg-IjcZ7IERqoxPunah .cluster span{color:#333;}#mermaid-svg-IjcZ7IERqoxPunah div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-IjcZ7IERqoxPunah :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;}

selector 版

selector

thread

channel

调用 selector 的 select() 会阻塞直到 channel 发生了读写就绪事件，这些事件发生，select 方法就会返回这些事件交给 thread 来处理

2. ByteBuffer

有一普通文本文件 data.txt，内容为

1234567890abcd

使用 FileChannel 来读取文件内容

@Slf4j

public class ChannelDemo1 {

public static void main(String[] args) {

try (RandomAccessFile file = new RandomAccessFile("helloword/data.txt", "rw")) {

FileChannel channel = file.getChannel();

ByteBuffer buffer = ByteBuffer.allocate(10);

do {

// 向 buffer 写入

int len = channel.read(buffer);

log.debug("读到字节数：{}", len);

if (len == -1) {

break;

}

// 切换 buffer 读模式

buffer.flip();

while(buffer.hasRemaining()) {

log.debug("{}", (char)buffer.get());

}

// 切换 buffer 写模式

buffer.clear();

} while (true);

} catch (IOException e) {

e.printStackTrace();

}

输出

10:39:03 [DEBUG] [main] c.i.n.ChannelDemo1 - 读到字节数：10

10:39:03 [DEBUG] [main] c.i.n.ChannelDemo1 - 1

10:39:03 [DEBUG] [main] c.i.n.ChannelDemo1 - 2

10:39:03 [DEBUG] [main] c.i.n.ChannelDemo1 - 3

10:39:03 [DEBUG] [main] c.i.n.ChannelDemo1 - 4

10:39:03 [DEBUG] [main] c.i.n.ChannelDemo1 - 5

10:39:03 [DEBUG] [main] c.i.n.ChannelDemo1 - 6

10:39:03 [DEBUG] [main] c.i.n.ChannelDemo1 - 7

10:39:03 [DEBUG] [main] c.i.n.ChannelDemo1 - 8

10:39:03 [DEBUG] [main] c.i.n.ChannelDemo1 - 9

10:39:03 [DEBUG] [main] c.i.n.ChannelDemo1 - 0

10:39:03 [DEBUG] [main] c.i.n.ChannelDemo1 - 读到字节数：4

10:39:03 [DEBUG] [main] c.i.n.ChannelDemo1 - a

10:39:03 [DEBUG] [main] c.i.n.ChannelDemo1 - b

10:39:03 [DEBUG] [main] c.i.n.ChannelDemo1 - c

10:39:03 [DEBUG] [main] c.i.n.ChannelDemo1 - d

10:39:03 [DEBUG] [main] c.i.n.ChannelDemo1 - 读到字节数：-1

2.1 ByteBuffer 正确使用姿势

向 buffer 写入数据，例如调用 channel.read(buffer)调用 flip() 切换至读模式从 buffer 读取数据，例如调用 buffer.get()调用 clear() 或 compact() 切换至写模式重复 1~4 步骤

2.2 ByteBuffer 结构

ByteBuffer 有以下重要属性

capacitypositionlimit

一开始

写模式下，position 是写入位置，limit 等于容量，下图表示写入了 4 个字节后的状态

flip 动作发生后，position 切换为读取位置，limit 切换为读取限制

读取 4 个字节后，状态

clear 动作发生后，状态

compact 方法，是把未读完的部分向前压缩，然后切换至写模式

 调试工具类

public class ByteBufferUtil {

private static final char[] BYTE2CHAR = new char[256];

private static final char[] HEXDUMP_TABLE = new char[256 * 4];

private static final String[] HEXPADDING = new String[16];

private static final String[] HEXDUMP_ROWPREFIXES = new String[65536 >>> 4];

private static final String[] BYTE2HEX = new String[256];

private static final String[] BYTEPADDING = new String[16];

static {

final char[] DIGITS = "0123456789abcdef".toCharArray();

for (int i = 0; i < 256; i++) {

HEXDUMP_TABLE[i << 1] = DIGITS[i >>> 4 & 0x0F];

HEXDUMP_TABLE[(i << 1) + 1] = DIGITS[i & 0x0F];

}

int i;

// Generate the lookup table for hex dump paddings

for (i = 0; i < HEXPADDING.length; i++) {

int padding = HEXPADDING.length - i;

StringBuilder buf = new StringBuilder(padding * 3);

for (int j = 0; j < padding; j++) {

buf.append(" ");

}

HEXPADDING[i] = buf.toString();

}

// Generate the lookup table for the start-offset header in each row (up to 64KiB).

for (i = 0; i < HEXDUMP_ROWPREFIXES.length; i++) {

StringBuilder buf = new StringBuilder(12);

buf.append(NEWLINE);

buf.append(Long.toHexString(i << 4 & 0xFFFFFFFFL | 0x100000000L));

buf.setCharAt(buf.length() - 9, '|');

buf.append('|');

HEXDUMP_ROWPREFIXES[i] = buf.toString();

}

// Generate the lookup table for byte-to-hex-dump conversion

for (i = 0; i < BYTE2HEX.length; i++) {

BYTE2HEX[i] = ' ' + StringUtil.byteToHexStringPadded(i);

}

// Generate the lookup table for byte dump paddings

for (i = 0; i < BYTEPADDING.length; i++) {

int padding = BYTEPADDING.length - i;

StringBuilder buf = new StringBuilder(padding);

for (int j = 0; j < padding; j++) {

buf.append(' ');

}

BYTEPADDING[i] = buf.toString();

}

// Generate the lookup table for byte-to-char conversion

for (i = 0; i < BYTE2CHAR.length; i++) {

if (i <= 0x1f || i >= 0x7f) {

BYTE2CHAR[i] = '.';

} else {

BYTE2CHAR[i] = (char) i;

}

/**

* 打印所有内容

* @param buffer

public static void debugAll(ByteBuffer buffer) {

int oldlimit = buffer.limit();

buffer.limit(buffer.capacity());

StringBuilder origin = new StringBuilder(256);

appendPrettyHexDump(origin, buffer, 0, buffer.capacity());

System.out.println("+--------+-------------------- all ------------------------+----------------+");

System.out.printf("position: [%d], limit: [%d]\n", buffer.position(), oldlimit);

System.out.println(origin);

buffer.limit(oldlimit);

}

/**

* 打印可读取内容

* @param buffer

public static void debugRead(ByteBuffer buffer) {

StringBuilder builder = new StringBuilder(256);

appendPrettyHexDump(builder, buffer, buffer.position(), buffer.limit() - buffer.position());

System.out.println("+--------+-------------------- read -----------------------+----------------+");

System.out.printf("position: [%d], limit: [%d]\n", buffer.position(), buffer.limit());

System.out.println(builder);

}

private static void appendPrettyHexDump(StringBuilder dump, ByteBuffer buf, int offset, int length) {

if (isOutOfBounds(offset, length, buf.capacity())) {

throw new IndexOutOfBoundsException(

"expected: " + "0 <= offset(" + offset + ") <= offset + length(" + length

+ ") <= " + "buf.capacity(" + buf.capacity() + ')');

}

if (length == 0) {

return;

}

dump.append(

" +-------------------------------------------------+" +

NEWLINE + " | 0 1 2 3 4 5 6 7 8 9 a b c d e f |" +

NEWLINE + "+--------+-------------------------------------------------+----------------+");

final int startIndex = offset;

final int fullRows = length >>> 4;

final int remainder = length & 0xF;

// Dump the rows which have 16 bytes.

for (int row = 0; row < fullRows; row++) {

int rowStartIndex = (row << 4) + startIndex;

// Per-row prefix.

appendHexDumpRowPrefix(dump, row, rowStartIndex);

// Hex dump

int rowEndIndex = rowStartIndex + 16;

for (int j = rowStartIndex; j < rowEndIndex; j++) {

dump.append(BYTE2HEX[getUnsignedByte(buf, j)]);

}

dump.append(" |");

// ASCII dump

for (int j = rowStartIndex; j < rowEndIndex; j++) {

dump.append(BYTE2CHAR[getUnsignedByte(buf, j)]);

}

dump.append('|');

}

// Dump the last row which has less than 16 bytes.

if (remainder != 0) {

int rowStartIndex = (fullRows << 4) + startIndex;

appendHexDumpRowPrefix(dump, fullRows, rowStartIndex);

// Hex dump

int rowEndIndex = rowStartIndex + remainder;

for (int j = rowStartIndex; j < rowEndIndex; j++) {

dump.append(BYTE2HEX[getUnsignedByte(buf, j)]);

}

dump.append(HEXPADDING[remainder]);

dump.append(" |");

// Ascii dump

for (int j = rowStartIndex; j < rowEndIndex; j++) {

dump.append(BYTE2CHAR[getUnsignedByte(buf, j)]);

}

dump.append(BYTEPADDING[remainder]);

dump.append('|');

}

dump.append(NEWLINE +

"+--------+-------------------------------------------------+----------------+");

}

private static void appendHexDumpRowPrefix(StringBuilder dump, int row, int rowStartIndex) {

if (row < HEXDUMP_ROWPREFIXES.length) {

dump.append(HEXDUMP_ROWPREFIXES[row]);

} else {

dump.append(NEWLINE);

dump.append(Long.toHexString(rowStartIndex & 0xFFFFFFFFL | 0x100000000L));

dump.setCharAt(dump.length() - 9, '|');

dump.append('|');

}

public static short getUnsignedByte(ByteBuffer buffer, int index) {

return (short) (buffer.get(index) & 0xFF);

}

2.3 ByteBuffer 常见方法

分配空间

可以使用 allocate 方法为 ByteBuffer 分配空间，其它 buffer 类也有该方法

Bytebuffer buf = ByteBuffer.allocate(16);

向 buffer 写入数据

有两种办法 : int readBytes = channel.read(buf); 和 buf.put((byte)127);

调用 channel 的 read 方法调用 buffer 自己的 put 方法

@Test

public void testByteBufferWrite() {

ByteBuffer buffer = ByteBuffer.allocate(16);

// 添加元素

buffer.put(new byte[]{'a', 'b', 'c', 'd', 'e'});

try {

// 切换读模式

// TODO : 读数据仅仅只是读, 不是取, 读之后, buf 中的数据依然存在

buffer.flip();

for (int i = 0; i < 4; i++) {

// 使用get时, 指针会一直从 position的位置向limit移动

byte b = buffer.get();

log.debug("{}", b);

// 可以调用 rewind 方法将 position 重新置为 0, 这样读取的一致是0位置的数据

// 或者调用 get(int i) 方法获取索引 i 的内容，它不会移动读指针

buffer.rewind();

}

} catch (BufferUnderflowException e) {

e.printStackTrace();

}

从 buffer 读取数据

同样有两种办法

调用 channel 的 write 方法调用 buffer 自己的 get 方法

int writeBytes = channel.write(buf);

和

byte b = buf.get();

get 方法会让 position 读指针向后走，如果想重复读取数据

可以调用 rewind 方法将 position 重新置为 0或者调用 get(int i) 方法获取索引 i 的内容，它不会移动读指针

mark 和 reset

mark 是在读取时，做一个标记，即使 position 改变，只要调用 reset 就能回到 mark 的位置

注意

rewind 和 flip 都会清除 mark 位置

/**

* TODO mark 是在读取时，做一个标记，即使 position 改变，只要调用 reset 就能回到 mark 的位置

@Test

public void testBufferWritePosition() {

ByteBuffer buffer = ByteBuffer.allocate(16);

// 添加元素

buffer.put(new byte[]{'a', 'b', 'c', 'd', 'e'});

try {

// 切换读模式

buffer.flip();

for (int i = 0; i < 30; i++) {

// 在初始的 position = 0 这个位置做一个标记在 0 ~ 2区间读取

if (buffer.position() == 0) {

buffer.mark();

}

log.debug("{}", (char) buffer.get());

// 重置

if (buffer.position() > 2) {

buffer.reset();

}

} catch (BufferUnderflowException e) {

e.printStackTrace();

}

12:20:48.193 [main] DEBUG cn.knightzz.netty.nio.buffer.ByteBufferMethod - a

12:20:48.196 [main] DEBUG cn.knightzz.netty.nio.buffer.ByteBufferMethod - b

12:20:48.196 [main] DEBUG cn.knightzz.netty.nio.buffer.ByteBufferMethod - c

12:20:48.196 [main] DEBUG cn.knightzz.netty.nio.buffer.ByteBufferMethod - a

12:20:48.196 [main] DEBUG cn.knightzz.netty.nio.buffer.ByteBufferMethod - b

12:20:48.196 [main] DEBUG cn.knightzz.netty.nio.buffer.ByteBufferMethod - c

12:20:48.196 [main] DEBUG cn.knightzz.netty.nio.buffer.ByteBufferMethod - a

12:20:48.196 [main] DEBUG cn.knightzz.netty.nio.buffer.ByteBufferMethod - b

12:20:48.196 [main] DEBUG cn.knightzz.netty.nio.buffer.ByteBufferMethod - c

12:20:48.196 [main] DEBUG cn.knightzz.netty.nio.buffer.ByteBufferMethod - a

字符串与 ByteBuffer 互转

@Test

public void testBuffer2String() {

// 字符串 => ByteBuffer

// 方式1

ByteBuffer buf01 = StandardCharsets.UTF_8.encode("你好");

// 方式2

ByteBuffer buf02 = Charset.forName("utf-8").encode("你好");

log.debug("{}", buf01); // pos=0 lim=6 cap=11

log.debug("{}", buf02);

// 解码

CharBuffer buf03 = StandardCharsets.UTF_8.decode(buf01);

log.debug("{}", buf03);

log.debug("{}", buf03.toString());

}

输出

12:21:25.244 [main] DEBUG cn.knightzz.netty.nio.buffer.ByteBufferMethod - java.nio.HeapByteBuffer[pos=0 lim=6 cap=11]

12:21:25.247 [main] DEBUG cn.knightzz.netty.nio.buffer.ByteBufferMethod - java.nio.HeapByteBuffer[pos=0 lim=6 cap=11]

12:21:25.248 [main] DEBUG cn.knightzz.netty.nio.buffer.ByteBufferMethod - 你好

⚠️ Buffer 的线程安全

Buffer 是非线程安全的

2.4 Scattering Reads

分散读取，有一个文本文件 3parts.txt

onetwothree

使用如下方式读取，可以将数据填充至多个 buffer

/**

* TODO 分散读取

@Test

public void testScatteringReads() {

String path = ByteBufferReadDemo.class.getClassLoader().getResource("data.txt").getPath();

try (RandomAccessFile file = new RandomAccessFile(path, "rw")) {

FileChannel channel = file.getChannel();

// 创建Buffer

ByteBuffer buf01 = ByteBuffer.allocate(4);

ByteBuffer buf02 = ByteBuffer.allocate(4);

ByteBuffer buf03 = ByteBuffer.allocate(4);

channel.read(new ByteBuffer[]{buf01, buf02, buf03});

buf01.flip();

buf02.flip();

buf03.flip();

while (buf01.hasRemaining()) {

log.debug("{}", (char) buf01.get());

}

log.debug("----------------------------------------\n");

while (buf02.hasRemaining()) {

log.debug("{}", (char) buf02.get());

}

log.debug("---------------------------------------- \n");

while (buf03.hasRemaining()) {

log.debug("{}", (char) buf03.get());

}

} catch (FileNotFoundException e) {

throw new RuntimeException(e);

} catch (IOException e) {

throw new RuntimeException(e);

}

结果

12:24:04.090 [main] DEBUG cn.knightzz.netty.nio.buffer.ByteBufferMethod - a

12:24:04.094 [main] DEBUG cn.knightzz.netty.nio.buffer.ByteBufferMethod - s

12:24:04.094 [main] DEBUG cn.knightzz.netty.nio.buffer.ByteBufferMethod - b

12:24:04.094 [main] DEBUG cn.knightzz.netty.nio.buffer.ByteBufferMethod - s

12:24:04.094 [main] DEBUG cn.knightzz.netty.nio.buffer.ByteBufferMethod - ----------------------------------------

12:24:04.094 [main] DEBUG cn.knightzz.netty.nio.buffer.ByteBufferMethod - c

12:24:04.094 [main] DEBUG cn.knightzz.netty.nio.buffer.ByteBufferMethod - s

12:24:04.094 [main] DEBUG cn.knightzz.netty.nio.buffer.ByteBufferMethod - 7

12:24:04.094 [main] DEBUG cn.knightzz.netty.nio.buffer.ByteBufferMethod - 8

12:24:04.094 [main] DEBUG cn.knightzz.netty.nio.buffer.ByteBufferMethod - ----------------------------------------

12:24:04.094 [main] DEBUG cn.knightzz.netty.nio.buffer.ByteBufferMethod - 9

12:24:04.094 [main] DEBUG cn.knightzz.netty.nio.buffer.ByteBufferMethod - 0

12:24:04.094 [main] DEBUG cn.knightzz.netty.nio.buffer.ByteBufferMethod - a

2.5 Gathering Writes

使用如下方式写入，可以将多个 buffer 的数据填充至 channel

/**

* 测试多个ByteBuffer合并

@Test

public void testGatherWrite() {

String path = ByteBufferMethod.class.getClassLoader().getResource("data.txt").getPath();

try (RandomAccessFile file = new RandomAccessFile(path, "rw")) {

FileChannel channel = file.getChannel();

ByteBuffer buf01 = ByteBuffer.allocate(2);

ByteBuffer buf02 = ByteBuffer.allocate(2);

ByteBuffer buf03 = ByteBuffer.allocate(2);

channel.position(14);

buf01.put(new byte[]{'a', 's'});

buf02.put(new byte[]{'b', 's'});

buf03.put(new byte[]{'c', 's'});

// 切换至写模式

buf01.flip();

buf02.flip();

buf03.flip();

channel.write(new ByteBuffer[]{buf01, buf02, buf03});

} catch (IOException e) {

e.printStackTrace();

}

2.6 练习

网络上有多条数据发送给服务端，数据之间使用 \n 进行分隔但由于某种原因这些数据在接收时，被进行了重新组合，例如原始数据有3条为

Hello,world\nI’m zhangsan\nHow are you?\n

变成了下面的两个 byteBuffer (黏包，半包)

Hello,world\nI’m zhangsan\nHow are you?\n

现在要求你编写程序，将错乱的数据恢复成原始的按 \n 分隔的数据

package cn.knightzz.netty.nio.exercise;

import io.netty.buffer.ByteBuf;

import lombok.extern.slf4j.Slf4j;

import java.nio.ByteBuffer;

import static cn.knightzz.netty.nio.buffer.ByteBufferUtil.debugAll;

/**

* @author 王天赐

* @title: TestExercise

* @projectName hm-netty-codes

* @description:

* @website http://knightzz.cn/

* @github https://github.com/knightzz1998

* @create: 2023-01-03 11:51

@SuppressWarnings("all")

@Slf4j

public class TestExercise {

public static void main(String[] args) {

ByteBuffer source = ByteBuffer.allocate(32);

source.put("Hello,world\nI'm zhangsan\nHo".getBytes());

split(source);

source.put("w are you?\nhaha!\n".getBytes());

}

private static void split(ByteBuffer source) {

// 切换至读模式

source.flip();

int oldLimit = source.limit();

for (int i = 0; i < oldLimit; i++) {

// 获取一条完整的消息, 然后进行存储

if(source.get(i) == '\n') {

// 将新的完整的消息存入ByteBuffer

// source.position() 是ByteBuffer起始索引, if 一条消息读完, position = 6

// 下一次的 source.position() 就是 6, 可以自己打断点调试下

int len = i + 1 - source.position();

ByteBuffer target = ByteBuffer.allocate(i + 1 - source.position());

// 从source读取, 向target写入

for(int k = 0 ; k < len ; k++) {

target.put(source.get());

}

// 打印target中的内容, 这里用了自己写的一个工具类, 可以自己写一个

debugAll(target);

}

// 将未读取的部分向左移动, 覆盖掉已经读取的, 为了和下面的数据进行拼接

source.compact();

}

好文链接

评论可见，请评论后查看内容，谢谢！！！

您阅读本篇文章共花了：

金钥匙

服务器 Netty网络编程 - NIO基础

python 深度学习人工智能 linux [Pytorch] timm.create

单元测试程序人生软件测试自动化测试职场和发展自动驾驶 | 仿真测试-HiL测试全解析

发表评论取消回复

金钥匙

服务器 Netty网络编程 - NIO基础

python 深度学习 人工智能 linux [Pytorch] timm.create

单元测试 程序人生 软件测试 自动化测试 职场和发展 自动驾驶 | 仿真测试-HiL测试全解析

相关文章

发表评论取消回复

python 深度学习人工智能 linux [Pytorch] timm.create

单元测试程序人生软件测试自动化测试职场和发展自动驾驶 | 仿真测试-HiL测试全解析