Description When installed, the FRED preference settings use defaults which are considered "safe" for a wide variety of computer hardware configurations. These default preference settings may adversely affect the performance of a distributed calculation when many rays are being traced, as the number of rays that can be simultaneously held in RAM is controlled by the Ray Buffer Preferences. When the number of rays allocated for RAM storage is small, excess rays are paged out to the hard disk in temporary buffer files and the data access from these files is slow compared to direct RAM access. If the remote instances are being run under the FRED Remote Service (via the rFredLogonUser account with "Log on as a service" rights), the default preference settings will be used by the remote node. Although the ray buffer settings can be tuned for each remote node's specific hardware capabilities through the ray_buf_frame_count and ray_buf_rays_per_frame members of the T_IPCINSTANCE structure (and the CSV configuration file), it can be simpler to just push the master node's preference settings out to each remote node. Be aware that this does not guarantee each remote node has the hardware capacity to support the preference settings of the master node, so the user should have some knowledge about the remote node configurations being used. If the remote instances are being connected to by MPI, then the preference settings may be applied to the registry, in which case they will be used for subsequent instances of FRED unless overwritten.
Keep in mind that some preference settings relating to random number generation, ARN usage, etc. have specific preference script commands that can be set from within the script(s) being executed on the remote nodes. The ray buffer preferences have no script access and may need to be varied in setup between remote nodes, making the ray buffer settings unique in this regard.
Syntax count = IPCPushPreferences( connections() )
Parameters count (Long) Returned number of active remote nodes that the master node's preference settings were pushed to.
connections() As T_IPCINSTANCE An array of T_IPCINSTANCE structures, with each structure defining the connection information for a remote node.
Example The example below demonstrates a typical distributed computing calculation where a FRED file (the one being run by the master) is pushed to the remote nodes, an embedded script within the FRED file is executed and FRED grid data (FGD) output files are pulled back to the master node for accumulation into a final result. Several helper functions are included to keep the code modular.
Sub Main
'This script demonstrates how to use the Distributed Computing component of FRED in order 'to send an analysis out to "remote" nodes on a Windows network, retrieve the results from 'each remote node and then recombine them on the "master" computer.
'Master document preparation ARNDeleteAllNodes() PrefsSetARNRetainCount( -1 ) ClearOutputWindow()
'Remove any FGD files in the current directory on the master node Dim nDel As Long nDel = KillFgd( GetDocDir() ) Print "" Print "Removed " & nDel & " FGD files from the current directory."
'Load the connections() array from a CSV file Dim nRemoteLoad As Long Dim connections() As T_IPCINSTANCE Dim configFile As String, configLocation As String configLocation = GetDocDir() & "\" configFile = "DCTestBed_Connections01.csv" nRemoteLoad = IPCLoadConfigFile( connections(), configLocation & configFile )
'Connect to the remote nodes Dim nConnect As Long nConnect = IPCConnect( connections() )
'Query the status of the remote nodes (info printed to output window). Dim nStatus As Long nStatus = IPCQueryStatus( connections(), "" )
'Push the preferences Dim nPref As Long nPref = IPCPushPreferences( connections() )
'Push the FRED file to the remote nodes (file is the same as the file being run on master) Dim nPush As Long, nDisconnect As Long Dim fName As String fName = name 'returns the name (with FRD extension) of the FRED document associated with this script nPush = IPCPushFile( connections(), fName, "", "overwrite copy" ) If nPush < 1 Then Print "No files were successfully pushed to the remotes. Terminating connections and stopping the master script." nDisconnect = IPCTerminate( connections(), "sterilize", "" ) End End If
'Save a configuration file for potential use later Dim nConSaved As Long nConSaved = IPCSaveConfigFile( connections(), GetDocDir() & "\DCTestBed_Reconnect.csv" ) Print "Saved " & nConSaved & " remote node connections to file."
'Load the model into the remote instances and execute an embedded script Dim scriptName As String, nScript As Long, nLoad As Long scriptName = "Remote Node Script" nLoad = IPCLoadModel( connections(), fName, "1d" ) nScript = IPCExecEmbeddedScript( connections(), scriptName, "1d" )
'Have the remote instances push FGD files back to the master node. Embedded script was 'written to produce FGD files. Dim nPull As Long Dim pulledFiles() As String nPull = IPCPullFile( connections(), "*.fgd", configLocation, "move", pulledFiles() )
'Print a report of the files that were pulled from the remotes Dim curFile As String Print "" Print "The following files were successfully pulled from the remote nodes:" For Each curFile In pulledFiles() Print Chr(9) & curFile Next
'Terminate the connection to remotes. nDisconnect = IPCTerminate( connections(), "sterilize", "" )
'Load FGD files from remotes into master FRED document and sum them together 'to form a final result. Dim nFgd As Long, finalArn As Long If nPull > 0 Then nFgd = LoadFgd( pulledFiles() ) Print "" Print "Loaded " & nFgd & " FGD files into ARNs." finalArn = CompositeAllARN( ) Print "Final result stored in ARN " & finalArn Else Print "No files were pulled from remote nodes. No final result generated." End If
Print "" Print "Distributed calculation completed."
End Sub
Function LoadFgd( ByVal in_files() As String ) As Long
'This helper function loops over an array of strings that 'specify the full file path to a set of FGD files and then 'loads them into the FRED document as ARN. 'INPUTS: ' in_files() = string defining the directory to be searched 'OUTPUTS: ' returns the number of ARN that were created
Dim cFile As String, splitStrs() As String Dim nF As Long
nF = 0 For Each cFile In in_files() splitStrs = Split(cFile, "\") 'split the full file path name by "\" splitStrs = Split( splitStrs(UBound(splitStrs)), ".") 'split the file name by "." to remove the file extension ARNCreateFromFile( cFile, splitStrs(0) ) nF += 1 Next
Return nF
End Function
Function CompositeAllARN( ) As Long
'This helper function loops over the Analysis Results folder 'and summs all of the ARN into a composite result. 'INPUTS: ' None 'OUTPUTS: ' Returns the node number of the composite ARN
'What node number is the last ARN on the tree at? Dim nArn As Long nArn = ARNGetMaxNodeNum()
'Make a copy of the zeroth ARN. This will become the 'final composite result node. Dim compArn As Long compArn = ARNCreateCopy( 0, "Composite Result" )
'Start looping over the other ARN starting at index 1 Dim curArn As Long For curArn = 1 To nArn ARNLinearCombination( 1, compArn, 1, curArn ) Next
Return compArn
End Function
Function KillFgd( ByVal in_dir As String ) As Long
'This helper function scans a directory for FGD files 'and deletes them 'INPUTS: ' in_dir = string defining the directory to be searched 'OUTPUTS: ' returns the number of FGD files that were deleted
'Change to the search directory ChDir( in_dir )
Dim nF As Long Dim cFile As String nF = 0 cFile = Dir$("*.fgd") While cFile <> "" Kill( cFile ) nF += 1 cFile = Dir$() Wend
Return nF
End Function
The example code above produces the following content in the FRED output window:
Removed 1 FGD files from the current directory.
[MASTER]IPCLoadConfigFile: 1 of 1 connections read from 'C:\temp\DCTestBed_Connections01.csv'
index active speed host status 1 true 100 computer1 ---
[MASTER]IPCConnect: 1 of 1 Remote FRED instances connected
index host result message 1 computer1 connected ---
[MASTER]IPCQueryStatus: 1 connections queried
index host message 1 computer1 idle
[MASTER]IPCPushPreferences: 1 of 1 active Nodes finished
index host status 1 computer1 finished
[MASTER]IPCPushFile: 1 files pushed from 'dctestbed.frd' to ''
index host src/xfer/dest message 1 computer1 1/1/1 finished
[MASTER]IPCSaveConfigFile: 1 of 1 connections written to 'C:\temp\DCTestBed_Reconnect.csv' Saved 1 remote node connections to file.
[MASTER]IPCLoadModel: 1 of 1 queued for loading file 'DCTestBed.frd'
[MASTER]IPCExecEmbeddedScript: 1 of 1 queued for execution 'Remote Node Script' (static load balancing)
[MASTER]IPCPullFile: 1 files pulled from '*.fgd' to 'C:\temp\'
index host src/xfer/dest message 1 computer1 1/1/1 finished
The following files were successfully pulled from the remote nodes: C:\temp\Color Image Result_computer1_lh_20160121162638_00000004.fgd
[MASTER]IPCTerminate: 1 of 1 Remote FRED instance terminations initiated
index host message 1 computer1 initiated
Loaded 1 FGD files into ARNs. Final result stored in ARN 1
Distributed calculation completed.
See Also Distributed Computing Script Commands
|