HOW to incorporate information from different sites and create your own unique content?

by KubiK
3 replies
  • WEB DESIGN
  • |
Hi all,

I have some idea of using the existing information provided from different major sites and incorporating them in order to provide end-users more useful and comprehensive information they need.

For example, using both Google map and a job search site such as Indeed.com in order to allow users to set up alerts to notice them jobs available in their selected proximity. I have a few technical and legal questions for this concept:

1) Legally, do I need to get Google and Indeed.com written consents for using their data? Or can I simply go ahead as their information are made public and my site will provide merged information that neither of them provides?

2) Technically, what programming skills do I need to perform these kind of information merging from various sites. Note: I would like to automate the process as much as possible.

3) Is it likely that these original sites will change markedly in terms of format and add-ons which can drastically impact on my resulting site in the future? If so, anyway to accommodate it?

4) Are there many existing sites already using these kind of concepts? If so, would you kindly point me to them?

Thanks. Any advice or comments will be greatly appreciated.
#content #create #incorporate #information #sites #unique
  • Profile picture of the author Mkeating3
    Hi Kubik,

    Let me take a crack at it since no one else has helped you out. My answers to your questions are next to the number that corresponds with your question.

    1. If you "scrape" the data from the front side of Google and also Indeed (if it is even possible) you will get away with it for a given period of time. Then if they find out (not sure how they can) they will most likely press a lawsuit or send a cease and desist letter. You will then need to stop and change your entire business model or shut down. There is the option of seeing if you can get access to their APIs which would make your life much easier because it will allow your site to communicate with the data that exists on their site. There is also the option to syndicate the jobs from Indeed to your site.

    2. If you aren't a programmer, you will be greatly wasting your time trying to figure out how to do this seeing as it isn't simple. You will need to get someone who has experience with php, xml, api's, html, css, mysql, and probably some other ones I don't know of since I'm not a skilled programmer.

    3. Not sure what you are asking here? If you are asking if they will change the format of the service you are trying to leverage (i.e. Map from Google and Jobs from Indeed) then there is no telling. They will usually update the integration method so third parties able to keep using it in the event there is a change, but if they decide to discontinue it then you will be forced to find an alternative method- This is a problem when relying on integration to third parties.

    4. If you are specifically referring to this statement "Google map and a job search site such as Indeed.com in order to allow users to set up alerts to notice them jobs available in their selected proximity", then I would assume most major job sites will allow you to do this. I know all site monster, careerbuilder, indeed, etc. all allow you to search within "x" amount of miles. But I don't know if they allow you to set up alerts for a type of job in a given radius.

    Just so you know, I'm not a lawyer or an expert in the area you are seeking information about. But I had a start up that dealt with rental properties and it was very easy to integrate properties from most major rental sites into my website. It was also easy to make these show up on a Google map. So I am almost positive what you are looking to achieve is doable, but the specifics, technical and legal, are something I can't give you.

    I hope this is helpful. Let me know if you have any other questions
    Signature

    Hit me up to collaborate :)

    {{ DiscussionBoard.errors[8076514].message }}
    • Profile picture of the author KubiK
      Mkeating3,

      Thanks very much for your insights. I would like to discuss with this more.

      Are you pointing that if I am using APIs instead of scraping, I am less likely to get into trouble and be able to integrate information from different sites without worrying the legal stuff? Also, what if I give them due credits? I look at "popular" site like Walkscore.com which is a site I presume not made by Google themselves, but by someone else who basically use what Google map offers and take it to another level by using an algorithm created by the Walkscore.com owner, I do see a disclaimer from Walkscore.com that they have used Google's data, does it mean it's OK without any repercussion?

      Interesting that you have mentioned your rental site, I am interested in integrating Google map with real estate-related information as well. Since you have been successfully integrating rental information from various sites into your own site, I wonder if you need to contact them or getting any consents in the first place?

      I have very basic experience in programming but not a programmer, but is willing to learn or outsource it depending on the depth needed.
      {{ DiscussionBoard.errors[8078973].message }}
  • Profile picture of the author Mkeating3
    Hi Kubik,

    Glad you find them helpful and I'd love to chat further (I'm passionate about this stuff).

    If you use a companies API to integrate information from their site into your own, then you will not have to worry about legal stuff. You just need to get permission to use it, which is very simple. On some services you just go to the developer section and it will tell you what you need to know. Here is Google's section for maps: https://developers.google.com/maps/ . For other companies like the rental companies I dealt with (i.e. Forrent.com), I just needed to contact them. Then they gave me all of the necessary documentation I needed to integrate their data onto my site. In either case, I don't know if you need the disclaimer. Walkscore may have put it there incase someone was to say "you said it was a 5 minute walk and it was 3 hours…"

    As for your programming ability, I am not an expert programmer and have never touched the API code myself, but in my opinion, I would outsource it. There are tons of people that have done this type of integration many times and would most likely be very cheap. However, if this isn't time sensitive and you want to learn, best of luck

    If you want some great resources I put together this internet marketing resources bundle that has a bunch of free guide, tips, etc. They are docs I have gathered in my Internet entrepreneur quest

    Shoot me a PM if you have more questions as I'd love to help out!

    Cheers,
    Mike
    Signature

    Hit me up to collaborate :)

    {{ DiscussionBoard.errors[8086127].message }}

Trending Topics