How to crawl Professional Networks companies in a single line of Python code

Enrich Layer API is a developer's tool. And the chances are that if you can code, you will have no problem scraping a website, unless it is Professional Social...

Enrich Layer API is a developer's tool. And the chances are that if you can code, you will have no problem scraping a website, unless it is professional networks. It is a full-time job keeping up with professional networks's bot detection and layout changes.

Enrich Layer has just gone live with an update that introduces a new feature – the ability to turn the platform Company Profile URLs into structured profile data. In this article, I will share how you can fetch structured data of the platform company profiles with no more than a long line of code.

The beauty of Enrich Layer API is that you do not have to concern yourself with the tedious task of building a professional networks scraper. Be it one profile or a million profile, today or three months from now, your code to get structured data of the platform Companies will remain the same one-liner python code.

What kind of data can you get by scraping the platform company profiles?

With Enrich Layer Company Profile Endpoint, you can get everything in the "About" page of a the platform Company profile. These include phone numbers, funding data, and office locations.

The following table will specify everything that the Enrich Layer Company Profile Endpoint will return:

Key Description

description The overview section in the About page

website The listed website on the profile

industry Industry of the company

company_size Listed range of company headcount

company_size_on_the platform Total employees on the platform that declared themselves to be staff of said company

hq Address of company's headquarters

company_type Enumerator of company type. Could be PUBLICLY_HELD or PRIVATELY_HELD

founded_year The year that this company is founded in

specialities List of specialities

locations List of locations

name Name of the company

tagline Tagline of the company

universal_name_id Month and year of the user's birthday (Dependent on profile's privacy settings)

profile_pic_url Profile picture of the

background_cover_image_url Wechat contact information (Dependent on profile's privacy settings)

funding_data Crunchbase data of said company's funding data

phone Phone number

html_src (Optional) HTML source of this professional networks Profile. To have this value shown, please include ` in the Enrich Layer request.

Example with code

Enrich Layer Company Profile Endpoint is like any other ReST API. Make a request, and get something back.

Here, I will make a request in Python with the requests library.

` from pprint import pprint import requests

api_

the platform_profile_

api_

header_ = requests.get(api_endpoint,

params={'url': the platform_profile_url},

headers=header_dic)

pprint(response.json()) ` In no more than 4 seconds, I will get back this result back in JSON format:

\{'background_cover_image_url': 'https://media-exp1.licdn.com/dms/image/C511BAQGaR4Ivd9F9-A/company-background_200/0? 'company_size': [201, 500], 'company_size_on_the platform': 569, 'company_type': 'PRIVATELY_HELD', 'description': 'Gojek is a Super App. It’s one app for ordering food, ' 'commuting, digital payments, shopping, hyper-local delivery, ' 'and two dozen services. It is Indonesia’s first and ' 'fastest-growing decacorn building an on-demand empire.\n' '\n' 'In the last 36 months, the startup’s total order volumes have ' 'grown to 1100x and diversified into 20+ verticals. The ' 'company runs the equivalent of three Indian unicorns rolled ' 'into one.\n' '\n' 'A total of 2,000,000 drivers collectively cover an average ' 'distance of 16.5 million kilometers each day, making Gojek ' 'Indonesia’s de facto transportation partner. Gojek is a verb. ' 'Gojek is a way of life. It is quite simply the operating ' 'system of Indonesia. 400+ engineers spread across Jakarta, ' 'Singapore and India make software decisions that impact ' 'entire Southeast Asia.\n' '\n' 'Gojek Tech is the product development and training center of ' 'Gojek. The tech team comprises of developers, data ' 'scientists, designers, and product managers who work on ' 'product innovation, mining data, and crafting consumer ' 'experiences. The average age of the team is 29 and it runs ' 'one of the largest JRuby, Java and Clojure & Go clusters in ' 'Asia.', 'founded_year': 2015, 'funding_data': \{'\$type': 'com.the platform.voyager.organization.FundingData', 'companyCrunchbaseUrl': 'https://www.crunchbase.com/organization/go-jek?utm_ Social Network&utm_ Social Network_companies&utm_ 'fundingRoundListCrunchbaseUrl': 'https://www.crunchbase.com/organization/go-jek/funding_rounds/funding_rounds_list?utm_ Social Network&utm_ Social Network_companies&utm_ 'lastFundingRound': \{'\$type': 'com.the platform.voyager.organization.FundingRound', 'announcedOn': \{'\$type': 'com.professionalsocialnetwork.common.Date', 'day': 3, 'month': 6, 'year': 2020\}, 'fundingRoundCrunchbaseUrl': 'https://www.crunchbase.com/funding_round/go-jek-series-f--983516e8?utm_ Social Network&utm_ Social Network_companies&utm_ 'fundingType': 'SERIES_F', 'investorsCrunchbaseUrl': 'https://www.crunchbase.com/funding_round/go-jek-series-f--983516e8?utm_ Social Network&utm_ Social Network_companies&utm_ 'leadInvestors': []}, 'investorCrunchbaseUrl': 'https://www.crunchbase.com/organization/paypal?utm_ Social Network&utm_ Social Network_companies&utm_ 'name': \{'\$type': 'com.the platform.voyager.common.TextViewModel', 'text': 'PayPal'\}}, ]}, 'investorCrunchbaseUrl': 'https://www.crunchbase.com/organization/facebook?utm_ Social Network&utm_ Social Network_companies&utm_ 'name': \{'\$type': 'com.the platform.voyager.common.TextViewModel', 'text': 'Facebook'\}}], 'moneyRaised': \{'\$type': 'com.professionalsocialnetwork.common.MoneyAmount', 'amount': '375000000', 'currencyCode': 'USD'\}, 'numOtherInvestors': 0}, 'numFundingRounds': 10, 'updatedAt': 1592331991}, 'hq': \{'city': 'Bengaluru', 'country': 'IN', 'geographic_area': 'Karnataka', 'is_hq': True, 'line_1': 'Diamond District, Tower-B, 4th Floor', 'postal_code': '560008'\}, 'industry': 'Internet', 'locations': [\{'city': 'Bengaluru', 'country': 'IN', 'geographic_area': 'Karnataka', 'is_hq': True, 'line_1': 'Diamond District, Tower-B, 4th Floor', 'postal_code': '560008'\}, \{'city': 'Jakarta Selatan', 'country': 'ID', 'geographic_area': 'Jakarta', 'is_hq': False, 'line_1': 'Pasaraya Blok M, Jalan Sultan Iskandarsyah II No.1, ' 'RT.3/RW.1, ', 'postal_code': '12160'\}, \{'city': 'Sinagpore', 'country': 'SG', 'geographic_area': 'Singapore', 'is_hq': False, 'line_1': '8 Shenton Way, AXA Tower', 'postal_code': '068811'\}, \{'city': 'Gurgaon', 'country': 'IN', 'geographic_area': 'Gurgaon', 'is_hq': False, 'line_1': '1st Floor, Tower A, Building 8A, DLF Cyber Hub', 'postal_code': '122002'\}], 'name': 'Gojek Tech', 'phone': None, 'profile_pic_url': 'https://media-exp1.licdn.com/dms/image/C510BAQFYcIg1UROecg/company-logo_400_400/0? 'specialities': [], 'tagline': '', 'universal_name_id': 'gojektech', 'website': 'https://www.gojek.io'}

Managed service to scrape public user profiles

I wish there is more, but this is it. Enrich Layer manages the changes in the platform layout and bot detection, so all you have to do is make the above requests a million times with different companies and concern yourself with product design.

Leave the hard stuff to us.

API documentation can be found at #Professional Social Network-api and you can try Enrich Layer out immediately with 10 credits by entering your email at .