My Review of K12 CS Ed Framework: A Prelude to a Dagstuhl Seminar on CS Ed Assessment

February 14, 2016 at 11:49 am 19 comments

I’m sharing in this blog post my comments on the K12 CS Ed framework, to serve two purposes.

First, to remind you to do the review which you can start here: The review period ends Wed Feb 17 at 11:59pm PT. Please participate. If you disagree with what I write below, be sure to do the review yourself and counter what I say!

Second, I’m writing from Schloss Dagstuhl where I will be attending the first (to my knowledge) Dagstuhl Seminar on CS Education, on Assessing Learning in Introductory Computer Science (click here for more info). As the seminar leaders say in their introduction, “What is needed are shared objectives and assessment methods that enable more useful computing education research while providing guidance to those outside the area.” The K12 CS Framework is an attempt to define shared objectives, which are a necessary predecessor to effective measurement of learning.

Now, the seminar leaders also raise the research question, “What outcomes would we like to see in non-major courses that are not merely preparing students to write scripts needed on the job?” Providing students with the CS knowledge so that they can “merely prepare to write scripts needed on the job” is an important goal.  I’d agree with “in addition to” more than “merely.” Outcomes for non-majors will likely be radically different than for majors — which will be the focus of my poster presentation later this week.

Criteria for Concepts and Practices

Concepts and practices should…

  • have broad importance across the field of computer science

I agree. Breadth is good. It’s not important to cover the field of computer science. There is more in mathematics and science than can (or should) be covered in K-12 education.

  • be important and relevant to all students’ current and future lives

This is the most important criterion, in opposition to “be important to computer scientists.”

  • have potential for connections with other disciplines’ practices, such as Math and Science
  • be a useful tool for developing and illuminating more ideas in computer science.

The former is very important. The latter is much less so. CS is very new. I’m not sure that we have the right ideas to teach everyone. Given a choice, it’s much more important to teach everyone CS that helps them learn math and science and is useful to understand computing in their lives. I’d rather not (for example) teach hex and octal just because we use hex and octal in CS.

Core Concepts

Computing Devices and Systems

Overview: Computing systems include a broad range of devices that incorporate hardware and software to process information using a variety of inputs and outputs. The term is not limited to computers themselves, but rather includes many everyday objects that contain computational components that sense and act on the world. Complex systems are built from simple components that interact under the control of an operating system in an ever-changing world of technology.

This is quite nice. I have a couple of suggestions. Great research by people like John Pane at CMU has shown that novices think of programs as responding to events, being reactive (e.g., think apps, games, GUI interfaces). Thinking of computing as input-processing-output is an older model, difficult to understand, and less common today.

Are operating systems so important that they belong in the overview? Yes, students should know about operating systems, but they’re not key to CS. I’m thinking of Dan Ingalls’ famous 1981 quote in Byte magazine:

Operating System: An operating system is a collection of things that don’t fit into a language. There shouldn’t be one.

Computing devices require continual diagnosis, maintenance, and upgrades. Effectively maintaining and troubleshooting complex devices is not only a critical skill, but is also essential for sustainability and cost-saving efforts. Researching and applying solutions found in existing knowledge-base resources and recognizing patterns play an important role in troubleshooting.

I strongly disagree with that first statement. I run software that’s been unchanged for over 15 years. It’s a goal of software not to require continual diagnosis, maintenance, and upgrades. It’s worth discussing why software needs continual diagnosis, maintenance, and upgrades, but it is not a key concept and assumption of CS that it does.

A computing system is comprised of an integrated collection of computer components that work together in a hierarchy to process information. At its most basic layer, a computing system operates through binary calculations conducted by transistors; at more advanced layers, a computing system is capable of performing high-level tasks, including the ability to interact with other computing systems.

Transistors? Really? Everyone has to know about transistors?  Computing can be mapped to a wide variety of underlying mechanisms (see the Tinkertoy Tic-Tac-Toe computer).

An operating system allows computer components to work together. Without an operating system, computing devices have minimal, if any, utility. The complexity, capability, and compatibility of an operating system impacts the functionality of a computer system.

This is obviously a false statement. The Internet has no operating system, yet leads to many components working together to great utility.

This would be a great place to introduce systems thinking. What are ways to organize components? An operating system is one. Having a powerful set of guidelines that define interactions (like in the Internet) is another. Which works better?

Networks and Communication

Overview: With the very first computing devices, knowledge and information became a very powerful thing. Sharing of that early knowledge gained from computing was a challenge. Networking and communication systems were designed to facilitate the interconnecting of devices and sharing of information. The practices of networking and computing now encompass reliability, authentication, confidentiality, security, acceptable use, filtering, cryptography, and mitigating security concerns and system breeches. A computer or computing system can no longer exist in isolation with the age of the Internet, and with demand for immediate connectivity of large numbers of new devices, this is an area that will continue to see lots of growth and innovation. Without networks and communication, the rest of the computing world would be isolated and the speed of innovation would be much slower.

This is really quite nice — I don’t have any concerns.

Protocols are designed, refined, and selected based on the desired result of communication among devices. Protocols are developed or updated as additional devices need to be connected and the security concerns, regulation, and standardization of devices change.

I’d rather see this section be a statement about communication. Protocols are a set of agreements, made by human beings. They can be structured in a range of ways. This is a great place to connect to other systems that communicate.

Transmitted data may be passed between many devices before reaching its final destination. Computing devices choose paths to transmit data based on a number of factors including routing algorithms, distance, security, redundancy, speed, error handling, and amount of information supported.

The first sentence is absolutely an important idea for understanding the Internet. The rest goes way too deep to be useful for all high school students.

Sure, talk about how the Internet routes data. I think it’s even useful to talk about UDP, in the sense that data can get out of order and there has to be an effort to serialize it and guarantee its delivery. I love the idea of talking about routing so that we can explain by policy ideas like censorship or even “turning off the Internet” are both so hard to actually implement. It’s important to talk about the Internet as a collection of networks held together by guidelines making it complicated to change. It’s super important to point out that there is no master operating system of the Internet — which is a great place to tie into system ideas of decentralized thinking in science.

I’d like for every student to know about the Domain Name Server system. It’s a great example of mapping (how names are mapped to addresses). It’s an amazing example of a decentralized system. It’s useful to know that it’s a separate system, e.g., your “Internet” connection may be working, but you might not be able to get to Google if your DNS server is failing.

As the size and complexity of networks grow, so does the need for systems to prioritize, distribute, load balance, be redundant, perform Quality of Service, provide gateways, be resilient and adaptable; through the use of Access and Distribution Layer Switching.

I don’t see any part of this which is useful for all students to know. I can’t find any definition of what “Distribution Layer Switching” is.

Data and Information

Once collected, data can be stored using computers in a variety of ways. The choices we make about how that data is represented, organized, and physically recorded has impacts on cost, speed, and reliability, as well as accessibility and security.

You’re missing an important point here: that in the end, everything in the computer is represented in binary, usually bytes. Everything, and the computer does now know which byte is a character in a text, the red channel of a particular pixel, or part of an instruction to the virus or to the operating system. The notion that everything in the computer is a representation is a powerful idea. It’s useful because it allows us to understand how we could possibly get a virus from a picture or Word file.

Yes, there are cost, speed, and reliability implications, but those are much higher levels of abstraction and not useful to everyone. Not everyone makes cost estimates of data representations. Everyone gets viruses.

Data often needs to be transformed from its raw state to be easily understood. Data can be transformed through mathematical expressions, aggregation (sum/average of rows/columns), rearrangement, and visualization. The type of transformation can influence the people who view the data.

These are narrow notions of transformation. How about just talking about noise and error? People make mistakes with data. That’s a powerful idea that crosses disciplines.

Data about prior events can be used to predict future events based upon computational models of varying complexity. The accuracy of the prediction depends on the choice of factors and the amount and diversity of data used to produce the model.

This is counter to how science thinks about the world today. This is the billiard ball, Newtonian model of the universe. Let’s not tell students this.

Programs and Algorithms


Types allow programmers to think of problems in terms of data and variables. Programmers use abstraction to define new data types, combine data with operations, and hide implementation details. Collections of data or data structures provide simple interfaces coupled with specific efficiency properties. (introduced at earlier grade bands: data representation, primitive data types, and operations associated with types)

I’m worried about this assumption that predecessor concepts will be introduced in elementary school and then expanded upon in high school. We have very little CS in schools today. States will choose where to emphasize: Build up high school CS first? Elementary school CS first? For at least a decade yet, few students will have K-12 CS Ed.
What happens to high school students who don’t get the predecessor concepts? Just give up on them and wait for the elementary school students?

Control constructs determine when sequences of instructions are executed. Recursion is a control technique in which a procedure calls itself. This is appropriate when problems can be expressed in terms of smaller versions of themselves. Selecting from different control structures that can be used to solve the same problem introduces a tradeoff between runtime efficiency and code readability. (introduced at earlier grade bands: sequence, iteration, branching, events, nesting, and competing control structures)

Not everyone needs to know recursion.

I wish that we taught students a simple notional machine and described everything in terms of that. Here’s what I teach my CS students: A computer can only do six things:

  1. They can store data with a name(s).
  2. They can name parts of programs (instructions), and follow those instruction when commanded.
  3. They can take data apart.
  4. They can transform data into other forms.
  5. They can follow a set of instructions repeatedly.
  6. They can test data (is this true or not?), then take actions depending on what the result is.

Everything that I teach in Python, I relate to one or more of those six points. I make sure that I relate everything to a small, easily learnable notional machine.

I do not see why K-8 students should learn about nesting and competing data structures. This is an example of teaching things that are in CS, but are not useful in math and science.

There are several steps in the development process, including problem clarification, design, implementation, and testing. Teams creating computational artifacts must make important design decisions and iteratively refine them. Different programming environments have distinct resources and features. Selecting from different programming languages and libraries introduces tradeoffs between functionality, efficiency, design and implementation time, security, and personal experience or preference.

No, not everyone needs to know this level of a software development process.

Impacts of Computing

K-8 Progression

In early grades, students differentiate between responsible and irresponsible computing behaviors. Students learn that responsible behaviors can help individuals while irresponsible behaviors can hurt individuals. They examine legal and ethical considerations for obtaining and sharing information and apply those behaviors to protect original ideas. As students progress academically, they engage in legal and ethical behaviors to guard against intrusive applications and promote a safe and secure computing experience.

9-12 Statement

Laws impact many areas of computing in an effort to protect privacy, data, property, information, and identity. The legal oversight of computing involves tradeoffs; such laws can expedite or delay advancements and infringe upon or protect human rights. Ethical concerns also shape computing practices and professions. International variations in legal and ethical considerations should be examined.

This is a great point to larger systems issues of politics and history. Describe democracy as a similar set of tradeoffs and design considerations.

The seven practices of computer science are the behaviors and ways of thinking that computationally literate citizens use to fully participate in the modern data-rich and interconnected digital world. Students in grades K-12 should engage in all seven practices over each grade band with increasing sophistication over time.

1. Recognizing and Representing Computational Problems

2. Developing Abstractions

3. Creating Computational Artifacts

4. Testing and Iteratively Refining

5. Fostering an Inclusive Computing Culture

6. Communicating About Computing

7. Collaborating With Computing

I’d rather see 2 as Using Abstractions. Not everyone is going to Develop Abstractions. Developing Abstractions isn’t necessary for everyone. It’s a difficult and challenging thing for students to learn, but you can be effective at using computing without it.

Communicating About Computing

Computationally literate citizens use a variety of mechanisms to share information and insights about computer science. This includes communicating about their design processes, the elements and functionality of computational artifacts, and both the technical and societal implications of computational solutions. For example, they write clear comments on their code, document their work through technical writing, and create demonstrations that include visualizing multiple representations and account for the diversity of audiences members. They attend to precision by using language in contextually appropriate ways (for example, the term function has meanings in both mathematics and computer programming).

These are really programmer-centric ways of talking about computing. Can we open this up, or not list any at all? I like the idea of videos or even dance about computing, not just comments in code.


It’s a starting point. I hope that it gets revised a lot before going out to states for standards.

Mostly, I want the framework writers to emphasize what everyone needs to know and teaching computing as a generative set of ideas. We can use computing as a powerful lever for learning in a wide variety of areas. Don’t teach everyone things about computing that are not generative, that only teach about CS itself.

Entry filed under: Uncategorized. Tags: , , , , .

Marvin Minsky and understanding things in more than one way The need for feedback to improve undergrad CS teaching

19 Comments Add your own

  • 1. alfredtwo  |  February 14, 2016 at 11:58 am

    “be a useful tool for developing and illuminating more ideas in computer science.”

    I see this as meaning a base for learning more as time goes on but I will bring it up when we (I’m on the writing team for that section) take it on again.

  • 2. Raul Miller  |  February 14, 2016 at 1:48 pm

    My initial reaction, on reading this, was that the internet is a computing device in much the same way that computers are a transistor devices.

    Put differently: Quantitative differences, when large enough, become qualitative differences. Or, perhaps: all abstractions are approximations. Or, perhaps: useful ideas often contain [hopefully minor] flaws. Or, …

  • 3. ialee  |  February 14, 2016 at 1:57 pm

    Using abstractions is too weak. I believe all students can create abstractions (even really young children, for example see . In the context of modeling and simulation I’ve seen a wide range of students capable of developing agent based models as abstractions of the real world. Abstraction are fundamental in learning math, science, literature, history, art….

    • 4. Mark Guzdial  |  February 14, 2016 at 2:10 pm

      Of course everyone abstracts. Not everyone creates computational abstraction. I know of no evidence that all K-12 students are capable of creating new computational abstractions (e.g., defining new types, deciding how to modularize a system and implement it, etc.). Much the opposite — there is lots of evidence that most students do not master the basics of creating computational abstractions. (My book presents some of that evidence.) Yes, some students can. I don’t believe that all students can.

      But we don’t even have to go that far. Not all students need to create computational abstractions, even if they’re capable of it. All people who program need to use computational abstractions. Why do we have to go further, from what everyone needs to what you want people to achieve?

      Irene, you say “Using abstractions is too weak.” In what sense “too weak”? What level of “strength” is the framework supposed to achieve? What goal are you trying to achieve? Is the point of the K12 CS Framework to push beyond what research and practice currently know as possible? That isn’t what I thought that the framework is about.

  • 5. ialee  |  February 15, 2016 at 8:00 am

    Maybe we need to agree on a definition of a computational abstraction. In Project GUTS students regularly develop new agents, behaviors and interactions. They modularize a complex system into components and implement those components as models. Is this not a description of computational abstraction? See for a study that describes students developing abstractions.

    By “too weak” I mean that “using abstractions” may not allow students to think for themselves, rather they may be asked to trust abstractions developed by others (perhaps without critical inspection). While this may be okay as a starting point, I don’t think should not be an end point for HS. I’m measuring “strength” in terms of being empowered (and expected) to think for oneself.

    • 6. Mark Guzdial  |  February 15, 2016 at 8:26 am

      I agree that that’s creating computational abstractions. I don’t believe that you can teach that to all students in the US. We don’t know how. I believe that students can think for themselves while using others’ abstractions.

      More importantly, a framework should not be built on “may not,” e.g., students may not think for themselves. Frameworks should be built on what we know, what we have research evidence for.

  • 7. ialee  |  February 15, 2016 at 9:26 am

    Okay, I guess we can agree to disagree.

    I believe we can engage all students in the US in creating computational abstractions because we have observed it on the ground in action with a wide range of students across the US within afterschool programs (where there is self-selection) and during the regular school day (for all students in a classroom).

    We support and guide students up to creating computational abstractions in middle school using a Use-Modify-Create progression. In this progression students first use abstractions (in our case base models) created by others, then they inspect them (read the code for understanding, identify abstractions and assumptions made), then they modify the abstractions to reflect local conditions or their interests and run experiments using their models as experimental test beds.

    Furthermore, given time and expectation / challenge to study a local issue or scientific phenomena using agent based modeling, middle school students have created new computational abstractions.

    Perhaps all students can get to “use (understand) and modify” a computational abstraction. We will be studying it more but, thus far, we do not have evidence that this is too far a reach for students at the middle school level.

    • 8. Mark Guzdial  |  February 15, 2016 at 10:25 am

      Totally agreed — we can agree to disagree.

      What is worth discussing is the meta-level: what goes into a framework? How do we decide? I totally believe that you have been successful in guiding students to construct computational abstractions. I’ve seen evidence of others that have been less successful. What evidence is the bar for entrance into a framework? What is the goal of a framework — to define what can happen, to define what is the limit of what might happen, to define what everyone needs?

  • 9. Mark Guzdial  |  February 15, 2016 at 11:54 am

    I'm getting several off-line questions, "Surely you didn't mean that X shouldn't be in the framework, do you?"

    I probably think that X should be taught to kids, maybe even to all kids. I almost certainly want X taught in AP CSP.

    I don't want X to be required for all students. Think about the borderline cases. Think about the kids with cognitive disabilities, with low-IQ. Think about the kids for whom just getting a high school diploma/certificate is a struggle and an achievement, and getting one makes a significant contribution to their economic future. They can get a job with a high school diploma/certificate that they can't otherwise. Is X so important that you'd deny that kid a certificate/diploma if he can't learn it?

    I'm willing to include many of the items on the K-12 CS Ed Framework on that must have list. All kids must know that the Internet crosses many computers and routes messages around outages/censorship. All kids must know that a computer is just a machine and doesn't understand anything.

    For me: For all X to be in the CS Ed Framework, you need to show that the baseline needs it and that you can teach it to them.

    • 10. gasstationwithoutpumps  |  February 15, 2016 at 2:46 pm

      A proper framework does not contain only the bare minimum requirements, but contains what you expect the average kids to learn and aspirational goals for the top students. Of course, these different standards need to be clearly identified.

      There has been a tendency in the US to teach to the framework (for science, math, and reading anyway), so if the framework is set up to include only the bare minimum, then good programs are stripped down or defunded.

      A good framework identifies what topics must be covered and what depth of understanding is expected of a minimal pass, an average student, and a top student. Throwing out material that is not understandable by a someone with an IQ of 70 means that there will be nothing left of value for the average students, much less those in the top quarter of the class.

      Students with low IQs are getting high school diplomas in some states without being able to read or do 3rd grade addition and subtraction. I don’t see that being used as an excuse to lower the literacy and numeracy standards, so it should not be used to water the CS framework down to nonexistence either.

      • 11. Mark Guzdial  |  February 15, 2016 at 4:16 pm

        I’ve heard the suggestion that the framework be described in layers, and I’m all in favor of that. I agree that we want to have richness for the full range of students. I don’t want to have a minimum that isn’t attainable for most students. (I do have some concern that we don’t have enough of a research base to be able to identify the layers.)

      • 12. rdovi  |  February 15, 2016 at 4:27 pm

        We are having a lot of this debate locally – should AP drive all high school curricula? It’s is intended to be college level material.

        Also have concerns that some of the included topics will allow state departments of ed to interpret CTE classes as CS – for example this could be read to imply computer repair is a necessary part of cs, or networking classes are enough and no new cs should be added. Many of the folks using the framework at a policy level are not cs content experts and we must be very careful about what we inject.

  • 13. alfredtwo  |  February 18, 2016 at 8:58 am

    I wish the framework could do something like the CS 2013 document for higher ed. Specifically have some things that absolutely have to be in a CS curriculum. Some stuff that should be included at 90% and some stuff that is elective. That is something that would probably have had to be baked into the process earlier though. Also we’re dealing with a different beast (k-12 and undergraduate education are very different) than CS 2013 was.

  • 14. fgmart  |  February 20, 2016 at 1:53 pm

    Does it strike anyone but me that the overall framing of this project is technical and vocational, and thus uninspiring?

    Mark — your picking up on the transistor mention is but one example of this.

    The way that the top-five things that get named is narrow.

    It makes me think we are specifying content for a some sort of IT certificate rather than revealing a profound field of inquiry.

    Perhaps it stems from the “top-five” things being consider simply as “core concepts” rather than “strands” [of inquiry] (per many standards) or the expansive “Big Ideas” of CS Principles.

    Apropos to top-level naming, consider:

    (expansive) “Languages, Machines, and Computation” [1] vs.
    (technical) “Computing Devices and Systems” [2]

    (expansive) “Communication and Coordination” [3] vs.
    (technical) “Networking and Communication” [4]

    (expansive) “Abstraction and Design” [5]
    (expansive) “Creativity” [6]

    vs. nothing about process in K12CS’s top-five.

    In K12CS, process stuff is in Practices, which at first glance makes sense.

    But! Our process — the way we conceive, create, and iterate — is different from how others do it.

    This is a big part of what makes our field special, and it needs top billing so that people notice it.

    I know lots of smart and well-intentioned people have invested lots of time in this work (and will invest more).

    But I am worried that (however unintentionally) this project is aiming us down a path that collapses rather than illuminates what is powerful about our ways of knowing.


    [1, 3, 5] UK’s Computing at School
    [2, 4] K12CS draft framework
    [6] CS Principles

    • 15. techkim  |  February 21, 2016 at 8:42 am

      Mark – excellent review, thanks for sharing.

      Fred – I totally agree. CS4All has so much potential, but too often I see the need to be “technically accurate” trump the need to communicate clearly. I’m not talking dumbing down, but as you say being inspirational. The big thing I see missing from the framework is making it relevant to the young people it is aimed at, like Exploring Computer Science does.

      • 16. gasstationwithoutpumps  |  February 21, 2016 at 1:11 pm

        Frameworks aren’t “aimed at young people” as students never read them, nor are they expected to. They are aimed at teachers, education administrators, and politicians. The teachers need clear, specific, and jargon-free guidelines. Education administrators need buzzwords and gobbledy-gook, since that is their main strength, while politicians need inspiring quotes that have nothing to do with reality. It is not possible to satisfy all these demands simultaneously. Unfortunately, the needs of the administrators and politicians nearly always trumps the needs of the teachers, so most frameworks are full of impenetrable educatese with a few unrealistic sound bites for politicians. I think that there is an 95% chance that the CS4All framework will have these characteristics.

        • 17. fgmart  |  February 22, 2016 at 4:15 pm

          Kevin, i am not expecting these to be inspiring to kids. I want them to be inspiring to *me*.

          and to portray our field as intellectually deep — like math or physics.

  • […] are out for review now, when the Framework is just finishing the first round of public comment (see my review here).  The CSTA Standards have a different goal than the Framework, from my reading of the standards […]

  • […] as I was looking through my blogroll, I read Mark Guzdial’s review of the Framework. I wished I had been able to review the framework with a group – I didn’t feel that I […]


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Trackback this post  |  Subscribe to the comments via RSS Feed

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 9,052 other followers


Recent Posts

Blog Stats

  • 2,030,649 hits
February 2016

CS Teaching Tips

%d bloggers like this: