Skip to main content

Week 1

Week 1

 Environmental Sources of Big Data

The best example of Big Data the is provided by the environment is the weather and the changes within it. Supercomputers are used to create weather simulations in real time to analyse current weather and look for patterns, and furthermore use this data to predict the weather that is still to come. This data includes things such as air pressure, humidity and sea surface temperature. 

https://www.futurelearn.com/courses/big-data-and-the-environment/3/steps/420355


 Observing Earth from space with Big Data

Big data can also help humans observe the planet from space and choose to view select parts of it. Satellites in space can be used to 24/7 monitor the planet and can send back data such as weather or communications data. These can also be used for sending mapping data for earth and cartography data.

Google earth is an example of that, where the entire planet is available to be viewed thanks to the data that has been obtained from satellites

GOSAT is an example of a satellite that monitors the earth's data. specifically, this is the first satellite that has been specifically set up to monitor the amount of greenhouse gasses in the earths atmosphere

https://www.futurelearn.com/courses/big-data-and-the-environment/3/steps/420356


 Big Data Storage

CEDA stands for Center for Environmental Data Analysis and has been working with big data for a long time. it focuses on atmospheric data sent from satellites as well as simulations, observations and aircraft. Data is added to CEDA in many different formats, ranging from the most basic ways such as a physical disk being sent in the mail to using the internet to share files using the file transfer protocol. it also allows adjacent computers to analyse the data without having to move it between networks which reduces the chance of duplicated data.

JASMIN is an analysis infrastructure held by CEDA which is run for its users. it works with petabyte-scale amounts of data storage and again allows users to analyse and exploit CEDA's data.

Currently, over 1000 users user JASMIN on a regular basis, which allows them to log in and analyse the data remotely. 30,000 people access JASMIN to view the data catalog. It works on a huge range of projects such as earthquake monitoring, greenhouse gas emissions, etc

https://www.futurelearn.com/courses/big-data-and-the-environment/3/steps/420358


 Big Data computing 

It is important when dealing with large amounts of big data that a company has the adequate amount of power in order to deal with the amount of data coming in and process it accordingly. Dr Jon Blower explains that "it is pointless having a super computer that can process all the data at light speed if it isn't able to actually move and read the data equally as fast" meaning that it is important to have all of these attributes balanced so that a company can take full advantage of the computers capabilities to handle the data appropriately 

https://www.futurelearn.com/courses/big-data-and-the-environment/3/steps/420361

Comments

Post a Comment

Popular posts from this blog

7. Limitations of traditional data analysis

7. Limitations of traditional data analysis Security: Security is foremost aspect for every technology. Big data is prone to data breaches. The important information that is provided to some third party may get leaked to customers. Proper encryption must be made in order to protect the data.  Large growth in data : data is growing faster than the processing power. Large volumes of data are being exploded in past years. We need some new machines to work; otherwise we will get over run by data. Large Data centres can solve this problem.  Inconsistencies: Sometimes the tools we use to gather big data sets are imprecise. This will happen when the data is collecting for example, consider Google search Edinburgh College results of the search on one day will be different from other day, this is mainly due to inconsistency in data collection.
Post a Comment
Read more

Week 3

Week 3 Big and small data Small data is a way of keeping the size of data down. A file such as an image could take a lot of space depending on the size of the image, but by compressing it and losing some of the quality you can massivly reduce ths size. An example of small data could be Meteorological Aviation Report (METAR). The image below shows this off, each section of the code represents a different piece of information and saves entire data entries from being typed out and takes less time to process.  https://www.futurelearn.com/courses/big-data-and-the-environment/3/steps/420387 Citizen Science Is the collection of data about the natural world that has been gathered by the general public, usually as part of a project with other data scientists. This can be used to gain vast amounts of information quickly since many people at once can gather the information as opposed to just one scientist or one sensor. One example is Thames 21, a charity which worked with Cit...
Post a Comment
Read more

2. Historical development of Big Data

2. Historical development of Big Data 1980's - networks started to appear across other countries, although possible, connection was still extremely difficult without travelling, the networks had to communicate in the same language as opposed to different ones. The principle link at CERN connected the US and Europe in 1989 1990's - remote access of data in the terabytes was now easily achievable around the world. To share data more easily, the web was created so that the actually location of the data wasn't necessary to know 2000's - Big data was now so large that CERN was unable to store such a vast amount of petabytes themselves and as such had to share it between partners around the world to offload some of it. This lead to the creation of cloud computing 2005 - The term Big Data is used for the first time by Roger Mougalas, a year after the creation of the term "web 2.0" which refers to data too large to handle with traditional business methods 2009 -...
Post a Comment
Read more