Versioning and Verification for your AWS Security Group Configuration

We strive to keep all aspects of our infrastructure managed by code instead of people; all of our servers are managed by Chef, Jenkins builds and tests our code, and so on. Additionally, we work to verify that all these things are behaving as expected – though Chef configures a service to run on a server, our monitoring server will verify that that service is, in fact, running. We wanted the same verification to make sure our AWS Security Groups are configured correctly and no unexpected changes have occurred.

This verification process is managed by the same service that watches all our other services for problems: Nagios. We check in a copy of our Security Group configuration to git and ask Nagios to verify the state of the live Security Groups against the checked-in version. If there is any discrepancy, Nagios will page us and alert us to the rule(s) that has/have been added or removed. The extra content field in the alert contains an easy-to-read statement about what changed. For example, “ec2 has an extra rule in the web security group: tcp/80-80 is allowed from 0.0.0.0.”

Beyond allowing us to use our existing code review processes for peer review, keeping the configuration in git provides us with an added benefit: we can add comments to the file with more detail about the reason behind a specific rule. No more will you have to remember that port 5666 is for NRPE; you can write a comment to that effect right there in the configuration file. While this isn’t too big a deal for standard services, it’s a huge benefit when adding custom or obscure services.

Since the AWS API will give you a dump of your Security Groups in JSON, it seemed like the easiest format to use for the configuration file in git. Unfortunately, JSON does not allow for comments (though it is lenient in differences in whitespace). So that we can have comments, the Nagios check will strip any line that matches /^ *#/ (any line that begins with a # character) before running it through the JSON parser.

To get started using this check,

  1. Grab a copy of check_aws_secgroups from our Ops repo on github. The check requires Python and the Boto library to talk to AWS. Run it once with -h to see the various flags to configure file locations.
  2. Set up an automated git checkout on your Nagios server. You’ll need to create an ssh key and a user on github that has read-only access.
  3. Configure it by putting your AWS credentials in /etc/nagios3/conf.d/aws.yml.

    :access_key_id: ABCDEFABCDEF0123456
    :secret_access_key: abcdef0123456789ABCDEF0123456789

  4. Get an initial dump of your Security Groups from EC2: check_aws_secgroups --dump-ec2 > /tmp/serialized_rules.json. You’ll probably want to rearrange and reformat the file to maximize readability, as well as add initial comments before checking this into git.
  5. Configure the Nagios check appropriately..

Getting an automated git checkout was a little harder than expected; there are a few variables that make it easier.

Here is the crontab entry:

*/10 * * * * export GIT_SSH=/var/lib/git_checkout/.ssh/git_ssh_wrapper.sh; export GIT_DIR=/var/lib/git_checkout/chef-repo/.git; export GIT_WORK_TREE=/var/lib/git_checkout/chef-repo; git fetch -q && git fetch –tags -q && git reset -q –hard origin/master

The git_ssh_wrapper.sh makes sure ssh won’t balk at github’s host key and gives the path to your read-only user’s ssh key.

#!/usr/bin/env bash
/usr/bin/env ssh -o “StrictHostKeyChecking=no” -i “/var/lib/git_checkout/.ssh/id_rsa” “$@”

Enjoy your deeper sleep, as you rest assured that your Security Groups really are what you know they should be!

Ben Hartshorne
April 29, 2013

Guest Post by James Sweet: How Parse Saved Me Time and Money

N-1-1It’s always refreshing to see mobile technology being used to help people. Today we have a guest post for you from developer James Sweet, who used Parse to build a prototype of a mobile app that helps people in emergencies by connecting them to nearby EMTs, medical professionals, and other first responders. He is based in Virginia, near Washington D.C.

His idea was recently selected as one of three finalists out of 2,000 competing companies to present their disruptive ideas at the The Economist’s Innovation Forum in Berkeley, California on March 28, 2013.

_________________________________________________________________________


How Things Came to Parse

The lowest offer I got from any mobile app development company to build my Minimally Viable Product was $10,000, and it would take at least a month to build. However, the only reason I considered contracting out some of my development work was because setting up a virtual server with a database and handling push notifications is a headache. I estimated the learning curve would take me months. 

Luckily, though, I found Parse through a mobile app development company called Citrrus.  Once I found Parse, I was able to build a basic version of my app in just over a week and at no monetary cost. Now that I know what I’m doing, I can build similar apps with Parse in a day.

How Parse Helped

  • For rapid prototyping Parse can’t be beat. They handle well solved problems that I would otherwise have to spend a lot of time troubleshooting. They even have built in point in polygon queries for sending push notifications to users in a geofence. New features are constantly being added to their SDK, and they are listening very closely to suggestions from the community.
  • Amazingly, their customer service is even better than Amazon’s. Whenever I’ve have a question, I talk to the same person each time. It’s been great not having to re-explain things to different people. I’ve always gotten a quick response from support and am rarely passed around to different departments or people.
  • Parse made it easy to try their platform without committing. I’ll have to acquire a LOT of users before I get anywhere near the push notification or data caps on my free account. Parse also lets me export my data to my own machine in JSON format whenever I feel like it.

About My App, N-1-1

My app is called N-1-1 and in an emergency it lets people alert nearby off duty EMTs, medical personal, and other responders who can quickly come to provide aid.

All the things needed for N-1-1’s MVP were built into the Parse SDK. Push notificatons, querying by geofence, user management, and an API.

 

Visit James’ site about N-1-1 here and view his presentation at The Economist’s Innovation Forum here.

Courtney Witmer
April 26, 2013

New Features in the Data Browser

Parse users love to store rich relational data on our backend. Today, we are releasing a number of features that will delight our users who make use of the relations.

The first change is that you can now create, update and delete relations in the data browser. For example, to model the “Like” relationship between “Comment” and “User”, developers can create a relation called “Likes” on the “Comment” class, and add the users that like the comment to the relation, all within the data browser.

relations

Second, one-to-many relationships are now links in the data browser, and clicking on the links will take you to the objects they point to. For example, you can click on the “author” link in the Post class to find out who wrote the post:

Screen Shot 2013-04-26 at 1.48.27 PM

The third feature allows you to delete the data in a field. In the past, to indicate that the data has been removed, developers have been making their fields null or empty, but what they really want is to completely delete the data. Now, this can be done. All you need to do is to right click on the data you would like to modify, and choose ‘delete value’ on the dropdown.

Screen Shot 2013-04-26 at 12.34.59 PM

We hope you like this set of features!

Andrew Wang
April 26, 2013

The Future of Parse

Parse has come a long way. In just under two years, we’ve gone from a rough prototype to powering tens of thousands of apps for a very broad spectrum of customers.

Some of the world’s best brands trust us with their entire mobile presence, and a growing number of the world’s brightest independent developers trust us with their next big thing. We couldn’t be happier.

As stewards of a good thing, we’re always thinking about the next step in growing Parse to become a leading platform in this age of mobile apps.

These steps come in all sizes. Most are small and incremental. Some are larger.  Today we’re excited to announce a pretty big one.

Parse has agreed to be acquired by Facebook. We expect the transaction to close shortly. Rest assured, Parse is not going away. It’s going to get better.

We’ve worked with Facebook for some time, and together we will continue offering our products and services. Check out Facebook’s blog post for more on this.

Combining forces with a partner like Facebook makes a lot of sense. In a short amount of time, we’ve built up a core technology and a great community of developers. Bringing that to Facebook allows us to work with their incredible talent and resources to build the ideal platform for developers.

We think this is the right way to accomplish what we set out to do. We’re excited about the future of Parse!

 

Here’s some quick Q&A we’d like to get out there.

Q: Will my Parse app be affected in any way? No.

Q: Will Parse apps have to use Facebook functionality? No.

Q: Will Parse honor my contract? Yes, of course.

Ilya Sukhar
Ilya Sukhar
CEO Parse Inc.
April 25, 2013

AnyYolk: A Parse-Powered HTML5 Game

It’s been truly amazing to see the tremendous breadth of apps that have been built on Parse. These talented developers have used our platform for everything from utility apps, to social networks, to games. But creating great apps is not without its trials. It requires weeks of research, hard work, and constant learning. At Parse we strive to make this process as easy as possible, not only with our powerful platform but also with in-depth sample apps and tutorials.

Today we are releasing AnyYolk, the newest member of our sample app family. AnyYolk is a web based game built using cutting edge HTML5 technologies. It’s built on top of Backbone.js, the same framework behind the Parse JavaScript SDK, does all of its animations using CSS3, and has a Parse powered backend to manage highscores.

We’ll keep updating AnyYolk with a bunch of cool gaming related features, so keep an eye out for future updates! You can find the source code on GitHub and play with a live version of the game at www.anyyolk.com (or right below).

We hosted a live tutorial on how to build AnyYolk using Parse, Backbone.js, and HTML5, which you can find the full re-cap for here.

Mattieu Gamache-Asselin
April 25, 2013

Webcast Recap: Cross-Platform Development for Windows Apps Using Parse

Thanks to everyone who attended today’s webcast, “Cross-Platform Development for Windows Apps Using Parse,” with David Poll, Software Engineer at Parse, and Matt Harrington, Developer Evangelist at Microsoft. This webcast explored developing against a Parse backend and sharing code between .NET, Windows 8, and Windows Phone 8. For anyone who would like a second look, or just in case you missed it, the full video is below.

Watch the recording:

Ashley Smith
April 24, 2013

Leviathan Games Revolutionizes Comics on iOS

Leviathan Logo

Return customers are incredibly important in our business. Not only is it a necessity for building a strong and profitable company, but it also means that you must be doing something right, which is what keeps the Parse team going strong. Keeping developers happy has always been our main focus, and we’re proud of the relationships we’ve built with some of the coolest development teams out there.

Leviathan Games is one such company building on the Parse platform. Leviathan has been working with top brands and IPs for more than a decade. In 2007, they beat out dozens of companies to be selected by Apple as one of the first 5 iOS developers, and worked for Apple and Disney on a “Pirates of the Caribbean” game that revolutionized gaming on the iPod. They are devoted to mobile games, specifically the freemium game model, and have never looked back.

Below, Wyeth Ridgway, President of Leviathan Games, tells us what brought them to Parse and why they’ve stuck with us.

 

Hi Wyeth! What can you tell us about Leviathan and your role there?

Leviathan create free-to-play games for mobile devices, including Apple iPads and iPhones, Android phones and tablets, and Amazon’s Kindle Fire. As President of the company, I focus on our long term strategic partnerships and business strategy.

Leviathan has put out a lot of great games; which of them utilize Parse?

All of the products we are currently developing or maintaining use Parse to some degree.  All of the games on our products page are using Parse except Smurfs, since it predated our adoption of the platform. Additionally, we have 3 unreleased products in development using Parse.

Where do the ideas for Leviathan’s apps come from?

For this I’ll focus on our “Bane of Yoto” app.

This interactive comic was inspired by our disappointment with the current comic offerings for mobile devices. All they are is scans of print comics – we knew we could do better. We created an interactive comic experience we call Dreamotion, which revolutionizes how comics are experienced. We also added a collectible card system that enables users to unlock additional content in the comic.

There is a video of the “making of” Yoto here, with a lot more information.

You must have a talented team, so what brought you to Parse?

I have been an advisor to many tech startups in the Boulder VC community. I had heard Parse first mentioned there, quite some time ago. I started tracking the tech and was amazed at how fast the feature iteration was taking place, and loved the general “pay by volume” business model. After 6 months or so, it was clear Parse would leave our current solution in the dust, so we made the transition.

Since making the switch, are there any benefits that you’ve noticed?

The scaling business and billing model are key. The rate at which new features rollout is also amazing. Parse just understands the ecosystem, where it fits in, and does what it needs to do perfectly.

You’re using Parse in a lot of your apps. What features are used and how are they integrated?

We use Parse to create user accounts linked with Facebook logins. We then store user data such as which cards the players collected, how much currency they have, and progress indicators that drive our metrics tracking. There is also a host of utility classes in the app powered by Parse. For example, every time someone crashes it posts a log of that crash and device information so our engineers can diagnose and fix the issue.

How many overall downloads have you seen for “Bane of Yoto”?

Between Yoto Episode 1 and 2, which launches soon, we should break 1 million downloads between Amazon, Google Play and iTunes.

Do you feel that Parse has been able to decrease the time your staff spends on development?

The best part about using Parse is that we used to have staff dedicated to maintaining our own servers, as well as writing and maintaining code that Parse now handles. It has let us focus our existing staff on our games, as opposed to infrastructure.

Push notifications are becoming quite popular; do you use this feature in your apps?

Yes, we use Parse to send notifications driving users to new products we launch, as well as informing them of product updates, sales, and so on.

What would you say is the main thing that keeps you using Parse?

Ease of use. Parse is a bunch of very smart people laser-focused on solving a scaling problem most developers can’t handle internally.

 

You can read more about Leviathan Games here, here, here, and here.

Courtney Witmer
April 22, 2013

Guest Post: A Brief History of the Cloud – And Why We Chose Parse

Inkstone

We’d like to introduce you to one of the awesome development teams building apps on the Parse platform, Inkstone. With clients like Priceline.com and KIA, we were honored to hear they wrote a post about what brought them to use Parse and thought we’d share their thoughts with you.

Written by Ryan Bruels, who works on iOS development at Inkstone, the original post (reprinted below) can be found here. Ryan spent five years on Apple’s software engineering team before leaving in 2008 to start DevToaster, an award-winning iOS app development studio. Now at Inkstone, he leads their iOS engineering team. When not at the office, Ryan can be found hacking electronics or hiking the beautiful Pacific Northwest with his camera.

——————————————————–

I’m going to take a moment to toot the horn for one of our favorite companies. Here at Inkstone, we connect all our apps with Parse, the rock-solid backend app platform. I want to explain a bit about why we chose them, and how we integrate our apps, but first I think it’s important to tell you a bit about “the cloud” and how it relates to mobile apps.

When Jay and I started our first iPhone development startup in 2008, we mostly built apps as standalone units—the user would launch the app, do his or her thing, and close the app. Many of them connected to network services, but largely in a read-only or “accessory to the web” fashion: checking the weather, reading a company’s catalog, etc. This was the way of things. A brief look at the top apps of 2008 validates the trend we saw. A lot of novelties and a lot of mobile clients to bigger services, but nothing terribly intelligent. But hey, virtual beer!

Flash forward a year and two billion apps (!) later. Quickly the world realized what a mobile app really was destined for—this wasn’t “just another client”. The smartphone was to become a platform unto itself, and in order for that platform to succeed, it needed to be much smarter. And given the constraints—processing power, network connectivity, memory, storage space—many of us realized the need to offload intelligence to backend servers that could handle the increased demands. And at least for mobile devices, “the cloud” was born.

In 2009 our company was tasked to build an ambitious social networking app. But existing systems to interlink mobile devices simply didn’t exist at the time, not with the features we needed. This was way beyond “data store”. So we called up our buddies, who were building huge Facebook apps at the time, and together we built a backend platform called Reservoir to handle storage, media processing, and real-time communication between iOS clients.

In 2009 we were pretty far ahead with Reservoir: We started right away with geo-loc and geo-search capabilities. We enabled iOS push notification support almost right as it launched, and had written a fairly robust iOS SDK by the end of the year. The data model was schemaless and easy for the iOS client to search and retrieve from. We were among the first companies to build these features and connect them into mobile apps. At the same time we were building Reservoir, many companies starting building similar systems. And within two more years, tons of apps had become very smart—push notifications, geo-location, rich data and high-performance queries all became standard and expected parts of the most popular apps.

In mid 2012 we discovered Parse, and have simply been blown away by it, again and again. It has everything we wished we’d implemented in Reservoir, and has lots we didn’t know we needed. No joke: within 3 months of writing our first Parse demo project, we set the shutdown date for Reservoir.

Parse enables all the features we need “in the cloud”, and does them across every platform we’d ever hope to touch. Just a brief sample of what they offer to developers like us:

  • Parse is exposed through a dead-simple SDK that minimizes complexity and promotes good design patterns. They also have a REST API when you want to get down to the metal.
  • Those SDKs exist for a ton of platforms: iOS, Android, Windows 8, Windows Phone 8, OS X, and JavaScript.
  • Security is handled strongly, simply and elegantly through class and object-level ACLs and judicious siloing of the various app & secret keys.
  • It supports user accounts—with all the bells and whistles you expect like custom accounts, account creation, password recovery, linking accounts across networks—and allows login via Facebook and Twitter. And it does all this in a way that takes up about ~10 lines of code in your app. Astounding.
  • Native features such as push notifications are tightly integrated, targetable to certain groups of users, and are wrapped on the backend with a solid analytics and management dashboard.
  • You can deploy custom server-side code modules to do important, secure, and/or expensive work outside the relatively insecure mobile app. We use this to handle date/time validation and security validation to protect transactions from anyone trying to hack the mobile apps.
  • There is a simple and pleasant web interface to manage your Parse instance that stays out of your face as much as possible.
  • Parse is extraordinarily cost-effective, particularly when comparing to building and rolling your own cloud solutions.

The Parse crew launches great new features almost weekly it seems, and they squash bugs with no mercy or quarter. While we have minor complaints about the system here and there, it’s no understatement that we’re in love, and that Parse has transformed the way we build apps.

Inkstone’s vision is centered around connecting mobile devices in the cloud, and we’re thrilled to have such an excellent platform to help us realize that. Contact us if you have questions about how we use Parse, or how we can use the cloud to make your mobile app even better!

Courtney Witmer
April 19, 2013

Parse Named to the Facebook Technology Partners Program

Blog-Post-fb

Today, we’re thrilled to announce that Parse has been named to the Facebook Technology Partners Program. Parse Social is an easy way to create rich social apps integrated with Facebook across multiple platforms like iOS, Android, HTML5, and more. With this announcement, we are redoubling our commitment to making Parse the best platform to power your Facebook apps.

Integrating Facebook with Parse Social

Parse Social provides out-of-the-box functionality that allows your users to utilize Login with Facebook and save and associate their information, all without the trouble of setting up your own backend. Using our native SDKs for iOS, Android, Windows 8, and JavaScript, you’ll be able to add Facebook functionality with just a few lines of code.

For example, this is how you would log in a user in iOS:

[PFFacebookUtils logInWithPermissions:permissions block:^(PFUser *user, NSError *error) {
    if (!user) {
        NSLog(@"Uh oh. The user cancelled the Facebook login.");
    } else if (user.isNew) {
        NSLog(@"User signed up and logged in through Facebook!");
    } else {
        NSLog(@"User logged in through Facebook!");
    }
}];

You can read more about login in our documentation.

Sample Apps to Get You Up and Running Fast

As part of the launch, we’ve also created a Parse + Facebook resource center which we will continually update with useful information about Facebook app development. We’ve got two great sample apps: Scrumptious and Anypic, which are real world samples of social apps.

Specifically, we’ve augmented Facebook’s Scrumptious app to store user and post data with Parse. Learn more about this by viewing the screencast below:

The Applauze App Uses Parse and Facebook

Parse-powered app Applauze is an event focused service brought to you by 955 Dreams, the makers of the wildly successful Parse-powered Band of the Day app. The app allows users to discover, share, and purchase tickets to great events and then sends users push notifications to remind them of the events for which they have registered. Applauze uses Parse Data as well as Facebook integration by Parse for social sharing and user login.

Download the Applauze app on the iTunes store here.

Go Forth and Build

Armed with Parse and Facebook, we’re confident that you’ll be able to build engaging apps in a fraction of the time it used to take you.

Mattieu Gamache-Asselin
April 17, 2013

Rapid Cloud Code Development with Parse Develop

In the past month, we’ve made several improvements to the parse command line tool to speed up the Cloud Code development process.

The biggest improvement is the new develop command that will monitor your project for any changes and automatically upload them to the Parse Cloud. The develop command will also stream log messages to the console as they show up in the Parse log. Running develop is as simple as:

$ parse develop MyDevelopmentApp
E2013-03-19:20:17:01.423Z] beforeSave handler in release 'v1' ran for GameScore with the input:
  {"original": null, "update":{"score": 1337}}
 and failed validation with Each GameScore must have a playerName
New release is named v58
I2013-03-19T20:17:10.343Z] Deployed v58 with triggers:
  GameScore:
    before_save

Many developers have one Parse app they use for developing new code and one Parse app to run their production code. To avoid accidentally running develop on a production app, the command line tool requires the app to be explicitly passed as an argument for develop.

We also improved deploys for projects with many cloud code files or people developing on slow connections by only uploading diffs.

And finally, we added a -f option to the tail command. This will stream log messages to your console as they appear in the Parse log, much like develop does.

With these improvements, Cloud Code development should be faster and more fun! For more information on develop, read the Development vs. Production section of the Cloud Code Guide.

Shyam Jayaraman
April 15, 2013

Archives

Categories

RSS Feed Follow us Like us