GeoChain: API Scraping with Ballerina

Ballerina makes it easy to scrape data from other endpoints. The example below uses some of the string manipulating capabilities discussed in String Manipulation with Ballerina to scrap a piece of data from one geonames API and using it calls another.

	import ballerina.net.http;
	import ballerina.lang.messages;
	import ballerina.lang.jsons;
	import ballerina.lang.system;
	import ballerina.lang.strings;
	function main(string[] args){
	string countryInfoQueryParams = "/countryInfoJSON?lang=en&country=" + args[1] + "&username=" + args[0];
	http:ClientConnector geoserviceEP = create http:ClientConnector("http://api.geonames.org");
	message m = {};
	//calling the country info endpoint
	message countryInfoResponse = http:ClientConnector.get (geoserviceEP, countryInfoQueryParams, m);
	json capitalJ = jsons:getJson(messages:getJsonPayload(countryInfoResponse), "$.geonames[:1].capital");
	string capitalS = strings:valueOf(capitalJ);
	string capitalParsed = strings:toLowerCase(strings:subString(capitalS, strings:indexOf(capitalS, "\"") + 1, strings:lastIndexOf(capitalS, "\"")));
	string wikiSearchQueryParams = "/wikipediaSearch?q="+ capitalParsed + "&maxRows=" + args[2] + "&username=" + args[0];
	//calling the wiki search endpoint
	message wikiSearchResponse = http:ClientConnector.get (geoserviceEP,wikiSearchQueryParams,m);
	json response = (json) messages:getXmlPayload(wikiSearchResponse);
	system:println(strings:valueOf(response));
	}

view raw GeoChain.bal hosted with ❤ by GitHub

To try out the example, first create an account with Geonames and activate free web service usage through ManageAccounts. Then run the ballerina with the command line parameters shown below,

ballerina <Geonames Username> <Country Code> <Number of Wiki hits to retrieve> 

example, 

ballerina testaccount1101 LK 20

Similarly, ballerina can be used for service chaining scenarios. Look to the Service Chaining example shipped with the distribution for more information. Find out more about Ballerina here: http://ballerinalang.org/


Note: this example was written for ballerina 0.87 and it may not work with other versions.