fcp.co logo transparent
fcp clapperboard
Welcome, Guest
Username: Password: Remember me
Forgot your password? Forgot your username? Create an account
Forum
FCP.Co Final Cut Pro Forum
Final Cut Pro X (FCPX)
Media & Cache on shared storage network
×

Please Update Your Avatar!

14 Nov 2020
Moving over to a new system, we couldn't take the old avatars - so please upload a new one!

TOPIC: Media & Cache on shared storage network

Media & Cache on shared storage network 05 Aug 2020 16:08 #109131

afite wrote: Here's a possible fix for folks like myself that don't have the same support as other NAS solutions like Lumaforge and SNS...

I have literally spent an insane amount of time researching ways to improve network performance in macOS Catalina and I have finally stumbled across a solution. I have documented it on Apple's community website.

https://discussions.apple.com/thread/251102946?page=1

Thanks so much for sharing.

I did some digging, to see what Lumaforge is doing for SMB with their so called "Magic" in their "Jellyfish Connect" app. As far as I can see, they simply change the following values in ~/Library/Preferences/nsmb.conf (can also be set for all users in /etc/nsmb.conf).
[default]
signing_required=no
protocol_vers_map=6
port445=netbios_only
I believe that port445=netbios_only setting is what Ronny was mentioning?

You can find more info about these settings in man nsmb.conf via Terminal.
port445 - How to use SMB TCP/UDP ports

"How to use SMB TCP/UDP ports" can be one of:

both - Attempt to connect via port 445. If that is unsuccessful, try to connect via NetBIOS.

netbios_only - Do not attempt to connect via port 445.

no_netbios - Attempt to connect via port 445. If that is unsuccessful, do not try to connect via NetBIOS.

"Bitmap of SMB Versions that are enabled" can be one of:

7 == 0111 - SMB 1/2/3 should be enabled
6 == 0110 -SMB 2/3 should be enabled
4 == 0100 - SMB 3 should be enabled

"Bitmap of SMB Versions that have signing required" can be one of:
7  Signing required for SMB 1/2/3.
6  Signing required for SMB 2/3.
4  Signing required for SMB 3.

They also set the following values for NFS and a few specific settings for those with Myricom 10 GbE adapters.

cat /etc/nfs.conf
# nfs.conf: The NFS Configuration File
# LumaForge Optimized
nfs.client.access_cache_timeout=30
nfs.client.nfsiod_thread_max=15
nfs.client.access_for_getattr=1
nfs.client.allow_async=1
nfs.client.mount.options=retrycnt=10,rwsize=65536,dsize=16384,readahead=128,rw,async
nfs.client.nextdowndelay=15
nfs.client.statfs_rate_limit=5
# End LumaForge Optimized
/etc/myri_settings.conf
net.myri10ge.en*.lro_cnt=8
net.myri10ge.en*.intr_coal_delay=5
They also check the NIC, and set 10 GbE NICs to Jumbo frames, if found. If it's 1 GbE, MTU is set to auto (1500).

I really don't understand why there is so much "secrecy" as to what different vendors are doing to solve the network performance issues in macOS. Can't we all just collaborate on getting the best performance from the macOS network stack, without hiding information from each other? What's the point?

ATTO on the other hand has created a free tuning tool for ATTO Network cards, called ATTO360. The actual settings they set for the different sysctl tuning profiles are as follows for 10 GbE FastFrame NICs, while there might be different settings for their new FastFrame3 10/25/50/100 NICs.

/etc/sysctl.conf
Writing settings in this file makes settings stick on reboot. If you want to see them "on-the-fly", you simply type the following command for each line. E.g.:
sudo sysctl -w net.inet.tcp.autorcvbufmax=2097152
High Throughput and Multi Stream Throughput
net.inet.tcp.autorcvbufmax=2097152
net.inet.tcp.autosndbufmax=2097152
net.inet.tcp.delayed_ack=0
net.inet.tcp.lro=1
net.inet.tcp.tso=1
net.inet.tcp.win_scale_factor=8
net.link.generic.system.hwcksum_rx=1
net.link.generic.system.hwcksum_tx=1
Low Latency
net.inet.tcp.autorcvbufmax=1048576
net.inet.tcp.autosndbufmax=1048576
net.inet.tcp.delayed_ack=0
net.inet.tcp.lro=0
net.inet.tcp.tso=4
net.inet.tcp.win_scale_factor=8
net.link.generic.system.hwcksum_rx=1
net.link.generic.system.hwcksum_tx=1
I'm actually seeing a huge difference in responsiveness wtih SMB read operations when net.inet.tcp.delayed_ack is set to 0, compared to the default value of 3 in macOS 10.15 Catalina.

ATTO is also explaining the different options in a PDF, which is pretty nice!
www.atto.com/pdfs/PRMA-0495-000.pdf

LRO – Large Receive Offload is a technique for increasing inbound throughput of high-bandwidth network conditions by reducing CPU overhead

TSO – TCP segmentation Offload is a technique for increasing outbound throughput of high-bandwidth network communications by reducing CPU overhead.

This partly explains why they set different values for "net.inet.tcp.lro" and "net.inet.tcp.tso", depending on whether you need extra Low Latency or extra High Throughput.

As always, be careful about what you change, and make a backup of your previous settings before changing.

Some tips regarding troubleshooting of SMB connectivity issues in macOS
In case anybody needs to troubleshoot SMB in macOS 10.3.6 and newer, these commands might reveal some interesting details/problems.

1. Enable SMB debug logging:
sudo sysctl -w net.smb.fs.loglevel=99
2. Start streaming log messages:
log stream --level debug --predicate 'senderImagePath contains "smb"'
3. Reproduce issues

You can also show log messages after testing with the following command:
log show --debug --predicate 'senderImagePath contains "smb"' --last 1h
When done testing, disable SMB debug logging in the end:
sudo sysctl -w net.smb.fs.loglevel=0

In some cases, you might need to capture some tcp packets as well, for further Wireshark analysis.

The commands above recently revealed some network memory (mbuf) issues with some 10 GbE macOS clients reading 2K dpx image sequences in Blackmagic DaVinci Resolve. The smb debug logs clearly showed that the SMB connection failed, and indicated sock_receivembuf being the root cause of the broken connection.
2019-03-06 14:41:34.564 Df kernel[0:398cd] (smbfs) nbssn_recv: Breaking connection, sock_receivembuf blocked for 20
2019-03-06 14:41:34.564 Df kernel[0:398cd] (smbfs) smb_iod_reconnect: Starting reconnect with server
2019-03-06 14:41:34.564 Df kernel[0:3992d] (smbfs) smb2_smb_parse_change_notify: smb_rq_reply failed 60
2019-03-06 14:41:34.564 Df kernel[0:398cd] (smbfs) tcp_connect: nbp_rcvbuf = 7405568 nbp_sndbuf = 7405568
2019-03-06 14:41:40.566 Df kernel[0:398cd] (smbfs) smbfs_down: Share not responding...
Simply increasing the mbuf to 512 MB solved the issue.

Running the Terminal command netstat -mm will also help tell you if you have any requests for memory delayed, which is a clear sign of “not enough memory”.

To change the mbuf value in macOS, boot into recovery mode (cmd+R during boot) and run the following Terminal command to increase it to 512 MB: nvram boot-args="ncl=262144"

The tip about increasing the mbuf value in macOS is explained in more detail in a recent Dell EMC Isilon macOS tuning doc: www.dellemc.com/resources/en-us/asset/wh...nce_optimization.pdf


Lastly, I'm also sharing some additional info regarding /etc/nsmb.conf, that I got from somebody in the Dell Isilon team a couple of years ago. Keep in mind that this was for macOS 10.13.6. However, it's still useful to see what the different options mean.
#
# /etc/nsmb.conf - Global Config
# ~/Library/Preferences/nsmb.conf - User Config, Overrides Global Config
# No reboot required, just unmount and remount
# man nsmb.conf - to view localized options
#
#
# Aug 28, 2018
# For macOS X 10.13 High Sierra
# Tested on macOS X 10.13.6 High Sierra
#
[default]
# Avoids any Kerberos or NTLM questions from the mac and assumes non-certed creds (standard in most cases)
# but if you have Kerb or NTLS you may want to set this to that - Speeds up mounting.
minauth=none
 
# NetBOIS on isilon is no good, this forces the mac to net even attempt NB and stricktly use DNS
port445=no_netbios
 
# Allows the mac to have synchronous R or W access to the same file within different SMB calls.
# Faster performance in some cases.
streams=yes
 
# Soft mounts, same concept as NFS soft mounts,
# offloads application hangs to the OS and decreases
# the chance of an application hang or crash because
# of some but below layer 4, i.e. lacp break, alt route, IO stall or busy, etc. - application stability
soft
 
# Change notifications are off. This is a lengthy one to explain but i'll try to summarize,
# macOS is not good with open file handles over SMB.
# This means when a user has been doing a bunch of work that day on the storage
# a bunch of lingering file handles will somewhat stay open,
# change notify is a part of the SMB stack which is a background communication
# between the mac and storage which will be chattier and chattier as the day goes on.
# This doesnt stop open file handles but it does stop the chattiness on the line
notify_off=yes
 
# This is the bit mode that forces SMB3 only connections and SIGNIFICANTLY decreases mount time from seconds to almost zero.
# If you have other non-SMB3 storage your users need to use another bit mode. you can find that in the man.
protocol_vers_map=4
 
# Already default now in 10.13 but i like it there in case Apple changes something again, which they do.
signing_required=no
 
# Checks that the negotition is infact the SMB level that is being requested
# by sending some test data that only the protocol can support.
# Slows things down, trust us, its SMB3.
validate_neg_off=yes
 
# Makes protocol stalls more apparent therefor making identifiable problems more apparent.
max_resp_timeout=1s
 
# How many simul requests can macOS make to ask find what is in an open finder window.
dir_cache_async_cnt=1000
 
# 0 means no max
dir_cache_max=0s
 
# 0 means no min
dir_cache_min=0s
 
# This is the ability to keep the cache and refer back to it.
# Having that enabled does not show any updates to the finder (finder refresh related).
# If the dir_cache is turned off then you make the request to fill the
# "temp" cache every time you jump into a folder instead of referring to a premade dir-cache.
dir_cache_off=yes
 
# Unknown, but helps some applications save/overwrite files ?
file_ids_off=yes

Please everybody, let's keep sharing our findings and stop hiding common knowledge like this.

Please Log in or Create an account to join the conversation.
Forum
FCP.Co Final Cut Pro Forum
Final Cut Pro X (FCPX)
Media & Cache on shared storage network