3 ms·

Show HN: MapReduce Implementation of Spatial Co-Location Mining Algorithm

43 points by shagunsodhani 10y ago

popey456963 10y ago
For someone with no prior knowledge, can someone explain what this is doing?
- danvoell 10y ago
  From the abstract - "Spatial association rule mining is a useful tool for discovering correlations and interesting relationships among spatial objects. Co-locations, or sets of spatial events which are frequently observed together in close proximity, are particularly useful for discovering their spatial dependencies. Although a number of spatial co-location mining algorithms have been developed, the computation of co-location pattern discovery remains prohibitively expensive with large data size and dense neighborhoods. We propose to leverage the power of parallel processing, in particular, the MapReduce framework to achieve higher spatial mining processing efficiency. MapReduce-like systems have been proven to be an efficient framework for large-scale data processing on clusters of commodity machines, and for big data analysis for many applications. The proposed parallel co-location mining algorithm was developed on MapReduce. The experimental result of the developed algorithm shows scalability in computational performance."
- coderdude 10y ago
  As someone else with no prior knowledge, I'll take a crack at it. Here's the paper: http://shagunsodhani.in/locis/docs/paper/paper.pdf http://shagunsodhani.in/locis/docs/paper/paper.pdf > Spatial association rule mining is a useful tool for discovering correlations and interesting relationships among spatial objects. Co-locations, or sets of spatial events which are frequently observed together in close proximity, are particularly useful for discovering their spatial dependencies. > Boolean spatial events (features) describe the presence of spatial events at different locations in geographic space. Examples of such data include disease outbreaks, crime incidents, traffic accidents, mobile service types, climate events, plants and species in ecology, and so on. See figure 1 in the paper for a visual. It sounds like the purpose of this is to find sets of events that occur close to each other, physically, using MapReduce over large amounts of data.
  - shagunsodhani 10y ago
    @coderdude, yes the purpose is to find sets of events that occur close to each other, physically, using MapReduce over large amounts of data. Thanks
  - SomewhatLikely 10y ago
    Paper link is broken, both in your comment and on the site.
    - shagunsodhani 10y ago
      Hi @SomewhatLikely. I have fixed it at both the place. Apologies for the confusion.
- shagunsodhani 10y ago
  Hey @popey456963, the implementation is to mine spatial colocation patterns. A spatial colocation pattern is a set of features that co-occur in space. For example, two crimes, say Robbery and Assault, would form a colocation pattern if they are reported together at many places. Think of spatial colocation pattern mining as association rule mining in the spatial domain. Thanks for asking this question, I have updated the page.

43 points by shagunsodhani 10y ago

popey456963 10y ago

For someone with no prior knowledge, can someone explain what this is doing?

danvoell 10y ago
From the abstract - "Spatial association rule mining is a useful tool for discovering correlations and interesting relationships among spatial objects. Co-locations, or sets of spatial events which are frequently observed together in close proximity, are particularly useful for discovering their spatial dependencies. Although a number of spatial co-location mining algorithms have been developed, the computation of co-location pattern discovery remains prohibitively expensive with large data size and dense neighborhoods. We propose to leverage the power of parallel processing, in particular, the MapReduce framework to achieve higher spatial mining processing efficiency. MapReduce-like systems have been proven to be an efficient framework for large-scale data processing on clusters of commodity machines, and for big data analysis for many applications. The proposed parallel co-location mining algorithm was developed on MapReduce. The experimental result of the developed algorithm shows scalability in computational performance."
coderdude 10y ago
As someone else with no prior knowledge, I'll take a crack at it. Here's the paper: http://shagunsodhani.in/locis/docs/paper/paper.pdf http://shagunsodhani.in/locis/docs/paper/paper.pdf > Spatial association rule mining is a useful tool for discovering correlations and interesting relationships among spatial objects. Co-locations, or sets of spatial events which are frequently observed together in close proximity, are particularly useful for discovering their spatial dependencies. > Boolean spatial events (features) describe the presence of spatial events at different locations in geographic space. Examples of such data include disease outbreaks, crime incidents, traffic accidents, mobile service types, climate events, plants and species in ecology, and so on. See figure 1 in the paper for a visual. It sounds like the purpose of this is to find sets of events that occur close to each other, physically, using MapReduce over large amounts of data.
- shagunsodhani 10y ago
  @coderdude, yes the purpose is to find sets of events that occur close to each other, physically, using MapReduce over large amounts of data. Thanks
- SomewhatLikely 10y ago
  Paper link is broken, both in your comment and on the site.
  - shagunsodhani 10y ago
    Hi @SomewhatLikely. I have fixed it at both the place. Apologies for the confusion.
shagunsodhani 10y ago
Hey @popey456963, the implementation is to mine spatial colocation patterns. A spatial colocation pattern is a set of features that co-occur in space. For example, two crimes, say Robbery and Assault, would form a colocation pattern if they are reported together at many places. Think of spatial colocation pattern mining as association rule mining in the spatial domain. Thanks for asking this question, I have updated the page.