linux/net/core
Eric Dumazet f161048060 net: use indirect call wrappers for skb_copy_datagram_iter()
commit 29f3490ba9 upstream.

TCP recvmsg() calls skb_copy_datagram_iter(), which
calls an indirect function (cb pointing to simple_copy_to_iter())
for every MSS (fragment) present in the skb.

CONFIG_RETPOLINE=y forces a very expensive operation
that we can avoid thanks to indirect call wrappers.

This patch gives a 13% increase of performance on
a single flow, if the bottleneck is the thread reading
the TCP socket.

Fixes: 950fcaecd5 ("datagram: consolidate datagram copy to iter helpers")
Signed-off-by: Eric Dumazet <edumazet@google.com>
Acked-by: Paolo Abeni <pabeni@redhat.com>
Acked-by: Willem de Bruijn <willemb@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2020-05-02 08:49:00 +02:00
..
Makefile
bpf_sk_storage.c bpf: Improve bucket_log calculation logic 2020-02-14 16:34:10 -05:00
datagram.c net: use indirect call wrappers for skb_copy_datagram_iter() 2020-05-02 08:49:00 +02:00
datagram.h
dev.c net: revert default NAPI poll timeout to 2 jiffies 2020-04-21 09:04:47 +02:00
dev_addr_lists.c net: remove unnecessary variables and callback 2019-10-24 14:53:49 -07:00
dev_ioctl.c
devlink.c devlink: validate length of region addr/len 2020-03-18 07:17:44 +01:00
drop_monitor.c drop_monitor: Do not cancel uninitialized work item 2020-02-11 04:35:51 -08:00
dst.c net: print proper warning on dst underflow 2019-09-26 09:05:56 +02:00
dst_cache.c
ethtool.c net: Zeroing the structure ethtool_wolinfo in ethtool_get_wol() 2019-10-26 11:20:10 -07:00
failover.c
fib_notifier.c net: fib_notifier: move fib_notifier_ops from struct net into per-net struct 2019-09-07 17:28:22 +02:00
fib_rules.c net: fib_rules: Correctly set table field when table number exceeds 8 bits 2020-03-05 16:43:31 +01:00
filter.c bpf: Return -EBADRQC for invalid map type in __bpf_tx_xdp_map 2020-02-24 08:36:48 +01:00
flow_dissector.c flow_dissector: Fix to use new variables for port ranges in bpf hook 2020-02-05 21:22:52 +00:00
flow_offload.c net: core: rename indirect block ingress cb function 2019-12-18 16:08:47 +01:00
gen_estimator.c
gen_stats.c
gro_cells.c
hwbm.c
link_watch.c net: link_watch: prevent starvation when processing linkwatch wq 2019-07-01 19:02:47 -07:00
lwt_bpf.c net: ipv6_stub: use ip6_dst_lookup_flow instead of ip6_dst_lookup 2019-12-18 16:08:42 +01:00
lwtunnel.c
neighbour.c net: neigh: use long type to store jiffies delta 2020-01-26 10:01:06 +01:00
net-procfs.c
net-sysfs.c net-sysfs: Call dev_hold always in netdev_queue_add_kobject 2020-01-26 10:01:09 +01:00
net-sysfs.h
net-traces.c
net_namespace.c netns: fix GFP flags in rtnl_net_notifyid() 2019-10-25 20:14:42 -07:00
netclassid_cgroup.c cgroup, netclassid: periodically release file_lock on classid updating 2020-03-18 07:17:38 +01:00
netevent.c
netpoll.c net: fix skb use after free in netpoll 2019-08-27 20:52:02 -07:00
netprio_cgroup.c
page_pool.c page_pool: do not release pool until inflight == 0. 2019-12-18 16:09:07 +01:00
pktgen.c net: Fix CONFIG_NET_CLS_ACT=n and CONFIG_NFT_FWD_NETDEV={y, m} build 2020-04-01 11:02:18 +02:00
ptp_classifier.c
request_sock.c tcp: add rcu protection around tp->fastopen_rsk 2019-10-13 10:13:08 -07:00
rtnetlink.c net: rtnetlink: validate IFLA_MTU attribute in rtnl_create_link() 2020-01-29 16:45:21 +01:00
scm.c
secure_seq.c
skbuff.c net: Fixed updating of ethertype in skb_mpls_push() 2019-12-18 16:08:56 +01:00
skmsg.c net, sk_msg: Don't check if sock is locked when tearing down psock 2020-01-29 16:45:31 +01:00
sock.c net: memcg: late association of sock to memcg 2020-03-18 07:17:43 +01:00
sock_diag.c sock: make cookie generation global instead of per netns 2019-08-09 13:14:46 -07:00
sock_map.c bpf, sockmap: Remove bucket->lock from sock_{hash|map}_free 2020-04-01 11:02:11 +02:00
sock_reuseport.c udp: correct reuseport selection with connected sockets 2019-09-16 09:02:18 +02:00
stream.c tcp: make sure EPOLLOUT wont be missed 2019-08-19 13:07:43 -07:00
sysctl_net_core.c net, sysctl: Fix compiler warning when only cBPF is present 2020-01-09 10:20:03 +01:00
timestamping.c
tso.c net: Use skb accessors in network core 2019-07-22 20:47:56 -07:00
utils.c net: Fix skb->csum update in inet_proto_csum_replace16(). 2020-02-05 21:22:52 +00:00
xdp.c xdp: obtain the mem_id mutex before trying to remove an entry. 2019-12-18 16:09:10 +01:00