Article Request Page ASABE Journal Article
Row-Crop Field Machinery CAN Bus Data Collection
Santosh Kumar Pitla1,*, CheeTown Liew1
Published in Case Studies and Modules for Data Science Instruction (): 1-9 (doi: ). Copyright American Society of Agricultural and Biological Engineers.
- Basics of Controller Area Network (CAN) bus systems and CAN bus data collection methods are presented
- CAN bus data message decoding and conversion into engineering units is demonstrated
- Data files and data analysis methods are made available for classroom use.
Abstract. Controller Area Network (CAN) bus systems are ubiquitous in modern on-road, off-road, and agricultural machinery. Today’s agricultural machinery utilizes multiple Electronic Control Units (ECUs) for performing various tractor and implement functions. These ECUs communicate with each other by exchanging CAN messages over a two-wire CAN bus system. Having a basic knowledge of CAN bus systems, CAN message structure and decoding, and CAN bus data collection methods are essential for extracting useful operational and performance information of row-crop agricultural machinery. This paper along with the supplemental data files provided can be used as an introductory resource to collect and interpret CAN bus data of agricultural machinery. This resource could be used as part of multiple laboratory exercises or hands-on activities for introducing CAN bus systems to students in both agricultural engineering and technology programs.
See this video:https://vimeo.com/435063466. Course materials are in a zip file for download.
Keywords.Machinery, Row-crops, CAN bus, Data, Analytics, Hands-on activities.
Learning Outcomes of This Exercise
- Demonstrate a basic understanding of CAN bus systems of agricultural machinery.
- Identify important fields of a CAN message frame.
- Use off-the-shelf CAN bus logging solutions for collecting CAN bus data from agricultural machinery
- Decode and convert raw CAN messages into engineering units using SAE J1939 standard document.
- Demonstrate the ability to use commonly available computer tools such as Microsoft Excel for importing and decoding CAN data files for analysis of machine operational characteristics.
Introduction and Background
CAN bus is a widely used technology for in-vehicle communication of on-road, off-road, and agricultural machinery (Stone et al., 2008). CAN is a multi-master broadcast type protocol where in all ECUs broadcast messages on a common bus. Each ECU is programmed to either receive or not receive certain messages available on the bus. ISO11898 standard provides information on the data link layer and physical layer of CAN (Kvaser, 2019; ISO 11898, 2015). SAE J1939 is a universal higher layer CAN protocol for heavy duty trucks, buses, planes, ships and mobile hydraulic machines (Kvaser, 2019). With SAE J1939 as its basis, ISO 11783 was developed to address the requirements of agricultural and forestry equipment (ISO 11783, 2017; SAE J1939, 2009; Voss, 2005; Walter and Walter, 2016).
Two types of CAN buses, tractor bus and implement bus, are available on modern agricultural machinery (see figure 1). Messages corresponding to engine and transmission control are typically found on the tractor bus, whereas, messages corresponding to GPS, hitch status, and implement status are found on the implement bus. Many off-the-shelf data loggers are available to collect CAN bus data from tractors and other agricultural machinery such as self-propelled sprayers and combines. Efforts to create open source CAN dataloggers are underway as well, however, the hardware and software needed to develop these loggers can be overwhelming for new users. ISOBlue is a good example of an open source project that provides details on how to develop CAN data loggers with telematics capabilities using open source software and hardware (ISOBlue, 2020).
Figure 1. Tractor and Implement CAN bus on a Tractor
These CAN data loggers have to be connected to ISOBUS diagnostic port available inside the machine cabs for data acquisition. The diagnostic port of the tractor and the Deutsch J1939 connector of the CAN data logger used for data collection in this exercise is shown in figure 2. The diagnostic port shown has six pins which correspond to two CAN bus channels (tractor bus and implement bus) and two pins for power. Each CAN bus channel is a two-wire (CAN_H and CAN_L) bus.
Figure 2. ISOBUS diagnostic port and data logger’s connector
Materials and Methods
A variety of off-the-shelf CAN loggers are available with different capabilities in terms of storage capacity, software sophistication, and remote connectivity. All three loggers shown in figure 3 have the capability to log data from both tractor and implement bus. Vector’s CANCaseXL box and Kvaser’s memorator use internal SD cards to store the data (see figure 3), whereas Farmobile data logger (Model: PUC, Farmobile, KS) is a telematics module that has the ability to stream data to a cloud for remote data acquisition and analytics. In this exercise we will use data files generated by Vector’s CANcaseXLlog data logger. The files generated by Vector’s CANcaseXLlog data logger are in .ASC format which could be easily accessed using a notepad or any text editor program. The files generated by Kvaser’s memorator device are also available in text file format, and hence we will just use the data file generated by Vector’s data logger to provide methods to convert raw CAN message data files into engineering units. The information deduced from Farmobile PUC data files will be summarized, however, detailed raw data file analysis are not presented to protect the identity and privacy of the user data.
Figure . (a) Vector’s data logger (Model: CANcase XL Log) (b) Kvaser’s data logger (Model: Memorator) (c) Farmobile data logger (Model: PUC)
A sampling from the raw data file obtained from Vector’s CANcaseXL log is shown in figure 4. This data was collected from the diagnostic port of a tractor performing test laps in a parking lot. Two important parts of a CAN message or CAN frame are 1) message identifier, and 2) data field with data bytes (see figure 4 and 5). Each CAN message can hold a maximum of 8 bytes of data. The message identifier and the data are represented in hexadecimal format as shown in figure 4. Each CAN message has a unique 11-bit or 29- bit message identifiers. Messages with 11-bit identifier are called standard CAN (CAN 2.0 A) frames, whereas messages with 29-bit identifier are called extended CAN (CAN 2.0 B) frames (see figure 5). Information such as priority of the message, parameter group number (PGN) and source address is encoded into the message identifier.
Figure 5. SAE J1939 CAN Message Identifier Structure
Message identifier CF00400 corresponds to PGN 61444 (where Click here to enter text. is equivalent toClick here to enter text., Click here to enter text. is in decimal format andClick here to enter text. is in hexadecimal format (Wolfram, 2021)) which belongs to the parameter group called “electronic engine controller 1” (SAE J1939, 2009). It contains important engine related information such as engine torque and engine speed which is encoded into different bytes and byte positions in the data field of the CAN message. SAE J1939 standard document (SAE J1939, 2009) have to be used to obtain data byte location and position information of engine torque and engine speed. Before the data conversion and manipulation steps, raw data file (figure 4) from Vector’s CANcaseXLlog which is in ASC file format has to be exported into programs such as MS Excel, MATLAB, R, or analyzed with Python code. We will use MS Excel in this exercise to convert data bytes in hexadecimal format into usable engineering units. A flow chart showing the steps needed to convert raw data files to interpretable information is presented in figure 6. The flow chart along with the supplemental files (File1: Vector CAN Data_FuelRate_EngineRPM_Torque.ASC; File2: Vector CAN Data_GPS.ASC; File 3: Torque_RPM.xlsx; File 4: Lab Exercise Instruction Sheet) provided, and the SAE J1939 standard document, will allow users to convert raw CAN bus data files into information in engineering units.
Figure . Steps for extracting useful information from a raw CAN data file
As a first step in the message decoding process, the data file (File 1) was imported into MS Excel and the desired PGN was used to filter CAN messages that were not needed for analysis (see also File 4: Lab Exercise Instruction Sheet, for importing process). In our example, PGN 61444 was used in the search criteria to filter out all other messages with different PGNs. The PGN 61444 was also used to search the SAE J1939 standard document for identifying the conversion factors required for converting data bytes in decimal format to engineering units. Based on the SAE J1939 standard document, engine speed is two bytes long and is located at fourth and fifth byte in the data field of the CAN message (see figure 7 and figure 4). To convert these bytes of data in hexadecimal format into engineering units, these two bytes of data will be combined and a conversion factor and offset value provided in the SAE J1939 document will be applied; in a y = mx + b “conversion equation” this conversion factor is the slope and the offset is the intercept. Two methods as shown in figure 7 can be used to convert hexadecimal data into engineering units (RPM in this case). Both methods use the MS Excel function “HEX2DEC” to convert the combined data bytes (Method 1) and individual data bytes (Method 2) into decimal format (see figure 7 and File4: Lab Instruction Sheet). A conversion factor of 0.125 (obtained from SAE J1939 document for PGN 61444) is multiplied with the decimal number to obtain engine speed in revolutions per minute (RPM). No conversion offset is needed for engine speed. Data bytes “22 2B” located at fourth and fifth byte positions in message “CF00400” translates to 1380.25 RPM engine speed after the conversion process. Method 2 utilizes a factor of 256^0 multiplied with the least significant byte (LSB) and 256^2 multiplied with the most significant byte (MSB) to convert the bytes of data in hexadecimal format into engineering units. It is important to note that the byte ordering of the LSB and MSB positions or the “endianness” in a CAN message is MSB last or so called little-Endiann (ISO 11898, 2015). It is in opposite direction when compared to the LSB and MSB positions used in conventional numbering system calculations. For example, for the CAN message shown in figure 7, the fourth byte of “22” is considered to be the LSB and fifth byte of “2B” is the MSB . The conversion factors obtained from SAE J1939 standard document and method 2 were used to convert vectors of raw CAN bus data into engineering units for analysis in the supplemental file (File 3:Torque_RPM.xlsx). As long as the byte-location, data conversion or scaling factors, offset values of user interested CAN messages are available in the SAE J1939 document, conversion of hexadecimal data into engineering units is possible. Equipment manufacturers also develop proprietary CAN messages for security purposes, in which case these conversion parameters may not be not available in the SAE J1939 standard documents. A small subset of other important engine and vehicle CAN message PGNs that are publicly available is presented in table 1. These PGNs can be used in the search criteria to find the byte positions, conversion factors, and offset values of parameters from the SAE J1939 standard document. For example, if a user is interested in knowing the byte positions, byte length, and conversion factors of a GPS CAN message, PGN “FEF3” could be used in the search field to obtain the required information. Similarly, “FEF2” PGN could be used for engine fuel rate information. The supplemental raw CAN data file (File 2: Vector CAN Data_GPS.ASC) contains GPS CAN messages that can be decoded to obtain GPS data in engineering units. The same methodology presented to decode engine speed messages can be used to obtain GPS data (latitude and longitude) in decimal degrees. Many other publicly available PGNs could be explored and decoded for additional analysis. For example, hitch position, hydraulic valve status, and ground speed information can be mined from CAN message streams. Tractor wheel slip and hydraulic power estimates, which are topics of interest for many agricultural engineering and technology curriculums, can be determined from CAN messages with minimal tractor instrumentation.
Figure . Converting CAN message bytes into engineering units
Table . CAN message information corresponding to Engine fuel rate, torque, speed and GPS
Results and Discussion
A plot of engine speed (RPM) and engine torque (%) obtained is presented in figure 8. Typically, CAN messages report percent engine torque instead of actual torque in the units of Nm. Rated engine torque information of the engine model installed in the machine can be used to convert percent torque into torque in the units of Nm. If the engine model information is not known, then CAN reported engine reference torque message with PGN 65251 could be used for determining the actual engine torque in Nm (Rohrer et al., 2018). The same method presented to convert engine speed from hexadecimal format into engineering units was used to convert engine percent torque in hexadecimal format into engineering units. However, one difference is that unlike engine speed data which is stored in two bytes in the CAN message, engine percent torque data is stored in only one byte. Byte 3 of the “CF00400” message contains engine percent torque data and hence there is no need of combining multiple bytes of data to obtain engine percent torque. The conversion factor to scale decimal number to percent engine torque is found in the SAE J1939 standard document. The supplemental excel sheet (File 3: Torque_RPM.xlsx) provided with this paper shows the raw CAN bus data conversion methods, conversion factors, and offset values.
Figure 8. Engine Speed (RPM) and Engine Torque (%) obtained after the conversion process
Variation in the engine speed and engine percent torque of a tractor during a test run in a parking lot can be observed from figure 8. Between approximately 18 and 22 seconds into the test run, peaks and drops in engine RPM and engine percent torque could be observed which correspond to operator throttle inputs. Figure 9 shows CAN reported GPS coordinates of planting operation with a 48-row planter unit. This data file is obtained from the Farmobile PUC (see figure 3C). Farmobile provides an output file with data in engineering units and hence they can be readily plotted for analysis.
In addition to GPS coordinates, some important data columns of interest are engine percent load, ground speed, and engine speed. These data columns were used to create a histogram of planter power (kW/row) in figure 10. The planter power values presented in figure 10 and figure 11 were calculated using the rated engine power information of the engine model of the tractor used for planting, and CAN reported (PGN 6144) engine percent torque and engine speed values. Planter power variation per row (kW/row) calculated based on CAN reported values in this field is presented in figure 11.
Figure 10.Frequency distribution of planter power requirements per row (kW/row)
MATLAB software was used to create this scatter plot with a gradient bar. Although MS Excel is an easy to use tool to get started with the basic CAN message decoding process, tools such as MATLAB, Python, and R with in-built functions and enhanced capabilities allow for comprehensive analysis of data. For example, MATLAB, Python and R have additional mapping and statistical capabilities which can be used to extract more insights and develop informative graphics. MATLAB’s in-built functions were used in this exercise to convert CAN reported GPS coordinates (latitude and longitude in decimal degrees) into UTM coordinates (northing and easting). From figure 10 and 11, it can be observed that the power requirements of the planter in parallel passes range from approximately 3.5 to 5.5 kWrow-1. For an average ground speed of 6.90 kmh-1, the power demand of the planter was found to be 4.15 kWrow-1.
Figure 11. Spatial variation of planter power demands
In addition to the power requirements of planters, fuel rate and other operational and performance parameters can be mined from CAN bus data files. For example, even without a GPS, by just observing the variation in fuel rate one can take a good guess of whether the planter is planting (implement engaged in the soil – increased fuel rate) or not planting (implement disengaged from the soil- reduced fuel rate). Improved estimation of field efficiencies and field capacities of the implements are possible with a simple analysis of some key publicly available CAN messages (Pitla et al., 2015). Operator interactions with equipment can be monitored in detail, and with some additional instrumentation and data mining, it might be possible to improve operator behavior for increasing field and machine efficiencies.
- CAN bus systems of agricultural machinery was introduced to provide a brief background and context for this CAN bus data collection and CAN message decoding exercise.
- Some examples of off-the-shelf CAN data logging hardware along with a raw CAN data file with messages in hexadecimal format was presented.
- CAN message frame corresponding to engine speed and engine percent torque was discussed in detail and the methods to convert hexadecimal data into engineering units using SAE J1939 standard document was presented. A supplemental MS Excel sheet detailing the conversion methods is provided.
- CAN reported planter power variation was presented to demonstrate the utility of CAN messages in determining in-field power demand estimation of agricultural machinery.
- This paper along with the supplemental data files provided could be utilized as a resource to introduce CAN bus systems of agricultural machinery to students in both agricultural engineering and technology programs.
The authors would like to thank Dr. Scott A. Shearer, Ohio State University, for providing the opportunity to collect tractor CAN bus data in the parking lot of the FABE department. The authors would also like to thank Nebraska producer, Kerry Knuth, for providing access to the CAN bus data of their planting operation.
International Organization for standardization. (2017). Tractors and machinery for agriculture and forestry — Serial control and communications data network — Part 1: General standard for mobile data communication. (ISO Standard No. 11783-1:2017) Retrieved from https://www.iso.org/standard/57556.html
International Organization for standardization. (2015). Road vehicles — Controller area network (CAN) — Part 1: Data link layer and physical signaling. (ISO Standard No. ISO 11898:2015) Retrieved from https://www.iso.org/standard/63648.html
ISOBlue (2020). What is an ISOBlue 2.0?. Retrieved from https://www.isoblue.org/docs/
Kvaser. (2019). J1939 Introduction. Retrieved from https://www.kvaser.com/about-can/higher-layer-protocols/j1939-introduction/
Pitla, S.K., Lin, N., Shearer,S.A., and Luck, J.D. 2015. Use of Controller Area Network (CAN) data to determine field efficiencies of agricultural machinery. Applied Engineering in Agriculture. Vol. 30(6): 829-839.
Pitla, S.K., Luck, J. D., Werner, J., Lin, N., Shearer, S. A. (2016). In-field fuel use and load states of agricultural field machinery. Computers and Electronics in Agriculture, 121, 290–300. https://doi.org/10.1016/j.compag.2015.12.023
Rohrer, R.A, Luck, J.D., Pitla, S.K., Hoy, R.M. (2018). Evaluation of the Accuracy of Machine Reported CAN Data for Engine Torque and Speed. Trans. ASABE. 61(5):1547- 1557
SAE International (2009). Surface Vehicle Recommended Practice - Vehicle Application Layer. Warren dale, PA: SAE International. SAE Standard J1939-71:2009. Retrieved from https://www.sae.org/standardsdev/groundvehicle/j1939a.htm
Stone, M. L., Benneweis, R. K., Van Bergeijk, J. (2008). Evolution of electronics for mobile agricultural equipment. Transactions of the ASABE,51(2), 385–390.
Voss, W. (2005). A Comprehensible Guide to Controller Area Network. Copperhill Technologies Corporation. ISBN: 978-09765116-0-1
Walter, E., Walter, R. (2016). Data Acquisition from HD Vehicles Using J1939 CAN Bus. SAE International. ISBN-10: 0768081726
Wolfram MathWorld, (2021). Hexadecimal. Retrieved from https://mathworld.wolfram.com/Hexadecimal.html
These files are in a zip file for download from the ASABE Technical Library:
File 1: Vector CAN Data_FuelRate_EngineRPM_Torque.ASC;
File 2: Vector CAN Data_GPS.ASC;
File 3: Torque_RPM.xlsx;
File 4: Lab Exercise Instruction Sheet.