A comprehensive list of answers to our most commonly asked questions.
Dairy Data Warehouse (DDW) is specializing in dairy farm data services for farmers, their farm consultants such as veterinarians, genetic consultants or feed advisers as well as companies serving dairy farmers across the globe. Our headquarters and customer services such as our multilingual helpdesk are based in Assen, Netherlands. While some of our developers are also based in Assen, most of them work from various locations mostly in Europe.Our mission: Providing access to high quality dairy dataWe live Dairy Data quality. High quality dairy data means for us unmatched level of reliability and richeness of the dairy data set.
Our Vision: Enable the dairy industry to unleash the power of BIG dairy data. We believe Dairy Data becomes most valuable, if a broad range of continuously updated and different data sources is combined and enriched with big data driven algorithms. We are passionate about the potential of BIG Dairy Data to transform modern dairy farming.
Data Access: The access to farm data often necessitates manual data export out of the farm management software and transfer to the vet via USB stick or email. This process is cumbersome, can be time consuming and particularly on larger farm operations, farm data becomes outdated quickly.
Data quality (rubbish in – rubbish out).): Farm management advice based on data relies on high quality, reliable farm data. For famers and their consultants, it can be difficult to assess the data quality of a farm data set. Advise based on inconsistent farm data can lead to wrong advice and actions on farm.
Data Comparability (Apples to apples): The comparison between farms in a similar situation can be a powerful tool to trigger action on farm. The comparability of farm data from farms using different data sources however is only possible using a diligent process of data transformation into a single, consistent and comparable data format.
Once the farmer has given DDW permission to connect their farm to the DDW dairy data platform, DDW transfers the farm data once a day from the herd management software on the farm PC to the DDW cloud.
The data is received on the DDW data platform, where the farm data is cleaned and converted. This unique and science driven process transforms all collected farm data from different data sources into a single, consistent data format stored in a single data base.
Dairy farm consultants with a DDW account can ask their dairy farm clients for access to their herd data. Only after explicit authorization by the farmer is it possible for the respective farm consultant to view the data on the DDW dairy data platform. The data can be viewed through different reports (milk production, reproduction, udder health) and web applications.
Data Extraction, Transformation (data conversion into a single, consistent format) and Loading (into a central data base, the Dairy Data Warehouse.
Process of cleaning up data in a database that is incorrect, incomplete, or duplicated.
Conversions are a set of algorithms that detect the type of farm management system the farm data set originated from and transform it into the DDW format. That makes all farm data inside the dairy data warehouse comparable. The conversion process is highly automated but remains under daily surveillance of a dedicated team. Our conversions are frequently updated to follow the constantly changing data base formats of the many data sources we transfer to the dairy data warehouse. Also, previously unknown data input errors of farmers and new scientific insights into dairy data handling are integrated into our formulas, to provide our customers with the highest dairy data quality.
Simply apply for a DDW user account by clicking 'Join Us’ and we will get in touch.
The DDW data marts are subsets of the big dairy data warehouse data base. Due to its smaller size a data mart allows faster execution of your queries compared to working directly on the central data base. Data marts are sold as “modules” on the DDW dairy data platform. Currently there the following modules available: Reproduction, Milkproduction, Udder health.
The data provider (in most cases this would be the farmer) remains owner of the data. The data owner retains therefore full control of the data. The data provider can ask DDW at any point to discontinue the data extraction of his farm data, by using the 'Contact Us' section at the top of this page. As a consequence, all consultants with access rights to the respective data will not be able to view these data any longer.
Anonymized data are farm data sets without any data that would allow identification of the data provider: for example address details, identification numbers of animals or names of inseminators. DDW uses anonymized data to offer services such as benchmarking.
Any consultant who wants to view farm data needs to send a data access request to the data provider. Only once the data provider has accepted the request can the consultant view the data on the DDW dairy data platform.
Data Upload Terms (DUT): The DUT contains in essence all the consents and approvals that are necessary from the farmer (or any other provider of data, where the data is collected by the DDW upload tool), ie. consent to install the DDW upload tool, transfer herd data, and share herd data with those the farmer consents to, as well as to use when anonymized by DDW.
The Herd Data Policy (the “Policy”) of Dairy Data Warehouse B.V. (“DDW”) sets out the policy commitment of DDW in relation to how herd data is accessed, gathered and used in the course of the DDW business. The Policy is intended to assist those who provide DDW with herd data (in the below referred to as “data providers”) and users of such herd data, to better understand the aims and intentions of DDW. The aspiration of DDW is that the Policy will clarify to data providers how their herd data will be used and to what extent herd data is made available to permitted users, and then with what limitations. Further, by the Policy DDW wants those that use the herd data provided by DDW to understand the rational behind any constraints DDW may have regarding the availability of herd data.
The principles set out in this Policy complements any legal requirements that may follow from laws or regulations and that applies to the operations of DDW.
Each data provider of DDW has consented to the Data Upload Terms. Whereas the Data Upload Terms contain the legal terms and conditions governing how herd data may be accessed, gathered and used by DDW, this Policy seeks to clarify how DDW collects and uses herd data in its operations. Such collection and use is of course always in compliance with the Data Upload Terms.
In this Policy, “herd data” means any type of data or information referring to the herd of a farmer, such as data regarding the identity of the farm, herd size and herd composition, milk yield, feed and feeding data, and data regarding health and breeding status.
Sources and Collection of Herd Data
DDW’s principal source of herd data is dairy farmers, globally. The herd data made available by famers, may also be complemented with herd data collected from dairy related organizations and authorities.
The collection of herd data is enabled by the DDW software application, which enables the DDW system to make a copy of the herd data available on the farm’s herd management system. DDW will never start collecting herd data from a potential data provider without the express consent from that data provider allowing DDW 1) access to its herd data, and 2) the right to install the DDW software application on the relevant IT-system of the data provider, all in accordance with the Data Upload Terms.
The collection of data by DDW is restricted to herd data (as described above). DDW will thus not access general business information, email correspondence, accounting information that is not directly related to the dairy herd, even if DDW would have technical possibilities to do so. If it would be discovered that DDW has such data in its systems for any reason, that data will be deleted immediately.
A data provider can at any time request (by email to firstname.lastname@example.org) that DDW’s connection to the farm management system of a data provider is terminated and that the collection of herd data is discontinued. We will comply with such request and commit to cut the connection within 48 h from such request.
All herd data is stored electronically on secure systems of DDW. The servers of DDW are currently located in Glinde, Germany, and satisfy customary safety, data security and scalability requirements.
Herd Data Use
DDW will only disclose herd data of a data provider to those expressly permitted by the data provider. If another business partner or other relation of a data provider requests access to the herd data of a data provider, such access will not be granted until the data provider has given its express consent to share its herd data with that business partner; relation. A data provider can at any time request DDW (by email to email@example.com), to discontinue the sharing of herd data to one or more of the recipients that a data provider has permitted herd data access to. Further, DDW reserves the right to at its sole discretion restrict release of herd data to certain recipients, if it would consider the release of such herd data be likely to jeopardise the future supply and collection of herd data.
In addition to the above, DDW may provide herd data statistics based on the herd data supplied by the data provider, eg. average somatic cell counts in a country, or average calving intervals in a specified region. Such herd data statistics may be provided to others than those expressly permitted by those data provider contributing to such herd data statistics, since herd data statistics are shared only in fully anonymized form. The requirements of DDW regarding anonymization go beyond legal requirements. DDW will not consider herd data anonymized – and thus will not qualify as herd data statistics – if it is possible to deduct or reverse engineer from that herd data statistics that certain data refers to or originates from a specific data provider.
DDW holds a copy of the herd data collected from the herd data providers, but does not claim ownership to that data. Any ownership rights, copyright or other intellectual property rights referring to the herd data, remains with the data provider. A data provider can at any time request DDW (by email to firstname.lastname@example.org) to discontinue the use of the herd data of that data provider. If so requested, DDW will discontinue the use of such herd data and that herd data will upon request be deleted.
Legal Obligations of DDW
Under the Code of Business Conduct of DDW, it commits to comply with the laws and regulations that apply to its operations, wherever it operates. In order to comply with this commitment, DDW will comply with any order or request from competent authorities that is given to DDW under applicable laws and regulations. So if DDW is requested by a local or governmental authority to provide herd data that DDW has in its custody, DDW will not supply such herd data to that authority unless it is proven that the authority is legally authorized to request such information and DDW by law or regulation is obliged to supply such information to the authority. If DDW is so legally obliged to provide herd data to the requesting authority, DDW will only disclose the data that it is legally required to and hence, will not go beyond what DDW is legally obliged to share by laws and regulations.
The above of course does not apply to sharing of herd data to an authority, if the sharing of herd data to such authority is expressly permitted by the data provider.
Assignment Rights of DDW
Under the Data Upload Terms, DDW reserves an assignment right to affiliates and other third parties. This right is reserved to cater for any legitimate future interests of DDW to perform corporate restructurings, outsourcings and similar rearrangements to its operations. Such potential assignment will not prejudice the rights of any data provider, in relation to any herd data available to DDW. If such assignments would occur, DDW commits not to disclose or extend user rights in relation to herd data to anyone not expressly authorized by the data provider.
If you have any questions regarding this Policy or the practices of DDW in relation to herd data access, gathering and use, please get in touch using the 'Contact Us' page.
Our business is founded on trust. If farmers don’t believe that we will always respect their ownership of the data they provide and take extreme measures to ensure the security and safety of their data, our model collapses. Reason enough for us to stick to our promise.