• How to convert two lists into a single list of tuples or vice versa?
  

  list1 = ["Hydrogen", "Helium", "Lithium"]
  list2 = ["H", "He", "Li"]
  list3 = [1, 2 ,3]
  
  element_list = Enum.zip(list1, list2) # [{"Hydrogen", "H"}, {"Helium", "He"}, {"Lithium", "Li"}]
  seperate_lists = Enum.unzip(element_list) # {["Hydrogen", "Helium", "Lithium"], ["H", "He", "Li"]}
  

1. It is used to process lists of tuples containing two elements that can be considered as “key and value” pair, where the key is an atom.
   
2. Elixir display them in keywork list format.
   

From lists to map: Keyword lists are a convenient way to address content stored in lists by key, but underneath, Elixir is still walking through the list. That might be OK if you have other plans for that list requiring walking through all of it, but it can be unnecessary overhead if you’re planning to use keys as your only approach to the data.

1. As key/value store
   

   # Dynamically sized map 
   empty_map = %{}
   
   # map with value
   squares = %{1 => 1, 2 => 4, 3 => 9}
   # or 
   squares = Map.new([{1, 1}, {2, 4}, {3, 9}])
   
   # fetch value for a given key 
   squares[2] # where 1 is the key, => 4
   squares[7] # => nil
   # or 
   Map.get(squares, 2) # => 4
   Map.get(squares, 8, :not_found) #=> :not_found, use this to specify default value.
   Map.fetch(squares, 5) #=> :error  This will distinguishly produce  
   # or raise exception 
   Map.fetch!(squares, 5)
   

2. Power dynamically sized key/value structures, overlap Tuples’ feature but let you to access fields by name
   

   bob = %{:name => "Bob", :age => 25, :works_at => "Initech"}
   # if keys are atom, we could short as 
   bob = %{name: "Bob", age: 25, works_at: "Initech"}
   
   
   # fetch 
   bob[:works_at] # => Initech
   bod[:non_exist_key] #=> nil 
   # or use short syntax if key is atom 
   bob.age #=> 25
   # though it will get KeyError if the key is not exist
   
   
   next_bob = %{bob | age: 26}
   # => %{age: 26, name: "Bob", works_at: "Initech"}
   next_bob = %{bob | age: 26, works_at: "no_where"} # change multiple key/value
   

   • Can only update keys exist in map!
     
   • Use map to hold structured data is very common.
     

From maps to structs: Structs are extensions built on top of maps that provide compile-time checks and default values.

defmodule Microsoft.Azure.Storage do
  @derive {Inspect, except: [:account_key]}
  defstruct [
    :account_name,
    :account_key,
    :aad_token_provider,
    :cloud_environment_suffix,
    :is_development_factory
  ]
  ...
end

• It is only possible to define a struct per module, as the struct it tied to the module itself
  
• Its fields:
  
  • could be a keyword list
    
  • or, a list of atoms as in this example: in this case, the atoms in the list will be used as the struct’s field names and they will all default to nil.
    
• About @derive
  
  • ref: derive/3
    
  • It is used with defstruct, see deriving.
    
• Here, we define a custom strcut and make it could be inspected, except :account_key field.
  

iex> <<0, 1, x::binary>> = <<0, 1, 2, 3>>
<<0, 1, 2, 3>>
iex> x
<<2, 3>>

• Matching on arbitrary length can only be done at end of the pattern and not anywhere else.
  

• If you have the data which can be arbitrary bit length then you can add bitstring instead, so the pattern now looks like.
  

  <<header :: size(8), data :: bitstring>>
  

iex> <<x::utf8, rest::binary>> = "über"
"über"
iex> x == ?ü
true
iex> rest
"ber"

• Therefore, when pattern matching on strings, it is important to use the utf8 modifier.
  

• Chunk format
  

  +--------------+----------------+-------------------+
  |  Length (32) | Chunk type (32)| Data (Length size)|
  +--------------+----------------+-------------------+
  |   CRC (32)   |
  +--------------+
  

• Pattern matching the chunk format
  

  <<length     :: size(32),
    chunk_type :: size(32),
    chunk_data :: binary - size(length),
    crc        :: size(32),
    chunks     :: binary>>
  

  • Another way of defining n byte length is binary - size(n).
    
  • Note: we matched length in pattern and used in the pattern as well. In Elixir pattern matching you can use the assigned variable in the pattern following it, thats why we are able to extract the chunk_data based on the length.
    

• create a project and start
  

  mix phx.new hello
  cd hello/
  cd assets/
  npm install
  cd ..
  mix ecto.create # create db configuration
  mix phx.server
  # or 
  iex -S mix phx.server
  

  • visit http://localhost:4000
    
• A new feature: print a string when you load a specific URL
  All actions is done in lib/hello_web
  
  1. Map requests coming in to a specific URL
     
     • Edit router, specify the controller, and a action name.
       
  2. tie a URL to a function on a controller
     
     • Define a function in controller
       
     • The name of the function should match the action name specified on router
       
  3. Tie that function to a view
     
     • But how how bind controller with view?
       
       • We defined a module HelloWeb.HelloView
         
       • That file doesn閳ユ獩 actually do any work beyond tying the view for world with some code to render a template. We閳ユ獟l rely on the defaults to render a template.
         
     • Inside the action function from controller, we specify the render with a template
       
  4. About request parameters
     
     • Defined in router
       
     • Extract out in controller with patter matching
       
       • Notice the convention from string to atom
         
     • Use it template(<action_name>.html.eex) with “@<parameter_name>”.
       

• About pattern matching, string and atom
  

  # from top to bottom, be shorthanded
  [{:name, name}]
  [:name => name]
  [name: name]
  name: name # if it is used as the last argument of a function
  

• Problems about starts project as iex -S mix phx.server from Emacs
  
  • Make sure the Emacs is running as Administrator.
    
  • If it has postgres issue, go to the project root, and re-run mix ecto.create.
    
  • When start Phoenix project from Emacs using alchemist, the default command is iex -S mix, we need to edit to iex -S mix phx.server
    

• How to check Phoenix version
  cd into mix project root folder, run
  

  mix phx.new --version
  

• A view in Phoenix is just a module, and templates are just functions.
  
• How to persistent out model data with database?
  
  1. Modify models from lib: for example, lib/rumbl/accounts/user.ex.
     
     • This is for schema definition.
       

  2. run ecto.gen.migration
     

     mix ecto.gen.migration create_users
     

     • This will create migration <timesteamp>_create_users.exs file in path priv/migrations/.
       
     • Edit the generated .exs file to define. This is for use code to operator database to create corresponding tables.
       
     • In general, we write elixir code to create/update table schema.
       
  3. run mix ecto.migrate
     
     • In this step, the actual table is created.
       

In general, 3 steps

• In lib modify our model
  
  • Define schema using Ecto.Schema
    
  • Define corresponding changeset.
    
• Generate migration file, in which we define database operations.
  
• Execute migration, by mix ecto.migrate.
  

• After
  
• Use mix phx.routes to check all available routes in our web application.
  

• After migration (create user table, we could test it from iex shell)
  

  alias Rumbl.Repo
  alias Rumbl.Accounts.User
  
  Repo.insert(%User{name: "Jose", username: "josevalim"})
  Repo.insert(%User{name: "Bruce", username: "redrapids"})
  Repo.insert(%User{name: "Chris", username: "mccord"})
  

• Check Phoenix.HTML.FormData contract to understand how the changes in the changeset available to the form.
  
• How to refer to specific routes in the application
  
  • Use YourApplication.Router.Helpers. In fact, phx help use with alias RumblWeb.Router.Helpers, as: Routes.
    
  • So, we can get any route through Routes.some_path.
    
• Install application as dependencies
  
  • edit mix.exs
    
  • mix deps.get
    
• Check point01
  
  1. Install password feature dependencies
     
  2. In user.ex
     
     • define schema for password and password_hash
       
     • create our registration_changeset
       

  3. Test in iex shell
     

     alias Rumbl.Accounts.User
     alias Rumbl.RumblWeb.Router.Helpers, as: Routes
     
     # this one is not valid 
     changeset = User.registration_changeset(%User{}, %{username: "max", name: "Max", password: "123"})
     
     # this one is valid 
     changeset = User.registration_changeset(%User{}, %{username: "max", name: "Max", password: "asecret"})
     

• Check point02
  

  • To fix already existing user to make them have valid password
    

    recompile()
    alias Rumbl.Repo
    
    for u <- Repo.all(User) do 
        Repo.update!(User.registration_changeset(u, %{password: "gghh3344"}))
    end
    
• Check point03
  
  1. At this point, all users shall meet the requirement: new user registration need to have passworld.
     
  2. In Account module, use User module’s registration changeset. (model)
     
  3. In user_controller, use exposed function from Account module.
     
  4. Modify new user html to provide slot for pasword. (view)
     
• Check point04
  Check if there is a new user in the session and store it in conn.assigns for every incoming request. In other words, we need to prevent user to access certain action when there is no session record.
  
  • loading data from session
    
  • use it to restric user access
    
• Check point05
  Add a mechanism to log the users in.
  
  • create login function in auth.ex
    
    • assigns current user
      
    • put session and configuration_session to reuse.
      
  • use login function in user controller
    

• Problem01
  

  lib/rumbl/accounts.ex:48: Users.__struct__/1 is undefined, cannot expand struct Users. Make sure the struct name is correct. If the struct name exists and is correct but it still cannot be found, you likely have cyclic module usage in your code
  

  • When I add a new user, it shows this.
    
  • Following error message, I could solve this: there is a typo in accounts.ex which I use %Users{} instead of should use %User{}.
    

• Check Point 01
  
  • Expose a function to validate username and password.
    
  • Use RESTful session API for
    
    • GET for new session login form
      
    • POST for login
      
    • DELETE for logout
      

• Check Point 02
  
  • Change the layout of the application to handle the new user features
    
• Problem01
  

  • When I go home to execute steps from check point02. I got error:
    

    iex(3)> for u <- Repo.all(User) do 
        Repo.update!(User.registration_changeset(u, %{password: "tmppass"}))
    end
    ...(3)> ...(3)> ** (Protocol.UndefinedError) protocol Ecto.Queryable not implemented for User of type Atom, the given module does not exist. This protocol is implemented for the following type(s): Atom, BitString, Ecto.Query, Ecto.SubQuery, Tuple
        (ecto 3.7.1) lib/ecto/queryable.ex:40: Ecto.Queryable.Atom.to_query/1
        (ecto 3.7.1) lib/ecto/repo/queryable.ex:16: Ecto.Repo.Queryable.all/3
    

    • Here, I forgot to refer User as alias Rumbl.Accounts.User. After execute alias Rumbl.Accounts.User. The problem solved.
      
• What is the differences between pipeline and plugs?
  

1. What is user registration
   
   • Apply changeset to Repo user.
     
   • User has username and password
     
2. What happended when user do a registration
   
   • We create a new(conn, _params_) in our controller to handle the get request to our url /users/new.
     
     • In which we use Accounts.change_registration which is an wrapper for user.registration_changeset.
       
       • In which the user’s params like username, passoword are validate by changeset and applyied with put_change.
         
3. What is the differences between new and create from user_{controller.ex}
   
   • new is used in controller to handle request to /users/new, it is used for rendering the form.
     
   • create is used in form Routes.user_path(@conn, :create), it is used for submiting the form.
     
4. What is login for a user
   
   • A user is login when the session contains the user’s username.
     
5. How to implement authentication feature (login and logout)
   
   • We implement authentication as a plug. So, we can add it to a plug pipeline for our router.
     
   • There are two kinds of plugs, one is function plugs and another is module plugs. When to prefer module plug over function plug?
     
     • When we want to share a plug across more than one module.
       

   • In module plug, there are two methods matters:
     

     • init
       
     • call
       
       • its second argument is the result of init.
         
       • its first argument is the conn which is Plug.Conn struct.
         

     So, we need to import import Plug.Conn.
     

   • The plug for authentication implementation:
     
     • Store the user ID in the session every time a user registers or a user login.
       
     • Check if there is a new user in the session and store it in the conn.assign for every incoming request.
       
       • Do this in our plug call.
         

1. What is a context?
   
   • A context in Phoenix is just a module that groups functions with a shared purpose.
     
   • A context encapsulates all business logic for a common purpose.
     
   • This way, we can interact with our business logic from controllers, channels, or remote APIs, without having to duplicate code.
     
   • In other words, a controller exists to work with context functions.
     
   • A controller parses end user requests, calls context functions, and translates those results into something the end user can understand. In other words, the controller’s job is to translate whatever our business logic returns into something meaningfull for the user.
     
   • The context doesn’t know about the controller, and the controller doesn’t know about the business rules.
     
   • When build a context, think about the way of how the context is available to the controller.
     
2. How to make a function plug available across controller and views?
   

   • In rumbl_web.ex import the plug function in both controller and router
     

     def router do
       quote do
         use Phoenix.Router
         ...
         import RumblWeb.Auth, only: [authenticate_user: 2]
       end
     end
     
     def controller do
       quote do
         use Phoenix.Controller, namespace: RumblWeb
         ...
         import RumblWeb.Auth, only: [authenticate_user: 2]
       end
     end
     

     • Where the 2 is the number of arguments expected by the authenticate_user.
       
3. What is the relationship between view and templates
   
   • A view pick all its corresponding templates and transform them into functions.
     
   • If a view is rumbl_web/view/video_view.ex, then the templates are located at rumbl_web/templates/video/.
     

4. How to find a video’s associated user without creating the bundling data:
   

   query = Ecto.assoc(video, :user)
   Repo.one(query)
   

   • Notice, here we avoid including a complete user info into video to find out this answer.
     
5. When building relationship between module, we generally to avoid having cyclic dependencies. That is, prefer the one-way relationship. Here, the Video schema depends on User.
   
6. How to use query to restrict CRUD operation of Video are limited to current user?
   
7. Problems
   

   • `Rumbl.Multimedia.Video` that was not loaded when try to associate a video with a user
     

     {:ok, video} = Rumbl.Multimedia.create_video(%{title: "new video", url: "http://example.com",
     
     alias Ecto.Changeset
     alias Rumbl.Repo
     
     user = Rumbl.Accounts.get_user_by(username: "zhaowei")
     changeset = video |> Changeset.change() |> Changeset.put_assoc(:user, user)
     

     • Error message
       

       ** (RuntimeError) attempting to cast or change association `user` from `Rumbl.Multimedia.Video` that was not loaded. Please preload your associations before manipulating them through changesets
       

       • The error message says the Video.user is not loaded.
         

     • Solution, we need to preload it:
       

       video = Rumbl.Repo.preload(video, :user)
       

       • The preload accepts one name or a collection of association names. After Ecto tries to fetch the association, we can reference the video.user. It is great for boundling data (we include a complete user info into the video).
         
       • Now, we could do the associate now.
         

     • At last, don’t forget to make changeset take effect
       

       video = Repo.update!(changeset)
       # check user binded to that video
       video.user
       

     • The above shows the steps to create an association between video and user. We could also do the following without the put_assoc.
       

       video = video
       |> Changeset.change()
       |> Changeset.put_change(:user_id, user.id)
       |> Repo.update!()
       
       video = Repo.preload(video, :user)
       

1. What is context generator
   Currently, we have met the following kind of generators
   

   1. mix.ecto.gen.migration, generate only migration files
      
   2. mix phx.gen.html, generate migrations, schemas, context, as well as controllers, views, and templates.
      
   3. mix phx.gen.context,useful for generating a resource with all of its context function.
      
   4. mix phx.gen.schema, useful for creating a resource when we want to define the context functions by ourselves.
      

   For more information, type mix help GENERATOR_NAME in the terminal.
   

2. How to add category into existing video?
   

   1. We choose to use mix phx.gen.schema to generate schema.
      

      mix phx.gen.schema Multimedia.Category categories name:string
      

      • We choose this because we probably don’t need most of the generated context function.
        
      • It produces two related files
        
        • category.ex
          
        • xxx_create_categories.exs. This file contains the migration which will create tables in db.
          
   2. Edit the generated migration file to fit our need.
      
      • Edit the “name” field as NOT NULL and create a unique index for it.
        
      • At this stage, we also edit the corresponding video schema from Video.ex to create a belongs-to relationship.
        

   3. Use mix ecto.gen.migration to generate a migration to add the category_id to our video table.
      

      mix ecto.gen.migration add_category_id_to_video
      

      • This command generate a migration with empty content left for us to fill.
        

      • Define the database contraint between videos and categories.
        

        defmodule Rumbl.Repo.Migrations.AddCategoryIdToVideo do
          use Ecto.Migration
        
          def change do
            alter table(:videos) do
              add :category_id, references(:categories)
            end
          end
        end
        

   4. Finally, migrate our database with our new migrations.
      

      mix ecto.migrate
      

   In general, we defines two migrations, one is to create categories table, another is to add the constrains on the existing video table.
   

3. How to regret a just did migration?
   
   • We could use mix ecto.rollback to migration down.
     
   • For example, we just did some migration. But we found we need to add an extra field for our just created table.
     
   • We could use mix ecto.rollback to revert the migration. Edit the change, then do mix ecto.migrate to apply the changes.
     
4. Seeding and associating categories
   
   • How to use script to populate our data while maintain database constrains
     
   • How to associate videos and categories
     
     • Fetch all categories name and IDs from db.
       
     • Sort them by name
       
     • Pass them into view as “select” input.
       
5. Problem: my categories currently don’t have unique value even when I already specify the unique contraint from schema.
   

   • The way I created table categories using migration
     

     defmodule Rumbl.Repo.Migrations.RecreateCategory do
       use Ecto.Migration
     
       def change do
         create table(:categories) do
           add :name, :string, null: false
     
           timestamps()
         end
     
         create unique_index(:categories, [:name])
       end
     end
     

   • The way how I add Category using changeset
     

     • In multimedia.ex
       

       alias Rumbl.Multimedia.Category
       
       def create_category!(name) do
         %Category{}
         |> Category.changeset(%{name: name})
       
         Repo.insert!(%Category{name: name}, on_conflict: :nothing)
       end
       

     • In category.ex
       

       defmodule Rumbl.Multimedia.Category do
         use Ecto.Schema
       
         import Ecto.Changeset
         import Ecto.Query
       
         schema "categories" do
           field :name, :string
       
           timestamps()
         end
       
         def changeset(category, attrs) do
           category
           |> cast(attrs, [:name])
           |> validate_required([:name])
           |> unique_constraint(:name)    
         end
       end
       
       

• How to delete all created categories
  
  • Currently, there are multiple duplicated values. I plan to delete all of them and create some values.
    

  • Delete existing categories
    

    alias Rumbl.Repo
    alias Rumbl.Multimedia.Category
    
    import Ecto.Query, only: [from: 2]
    
    query = Category
    Rumbl.Repo.delete_all(Category)
    

  • Populate category
    

    alias Rumbl.Multimedia
    
    for category <- ~w(Action Drama Romance Comedy Sci-fi) do
      Multimedia.create_category!(category)
    end
    

  • Why my changeset doesn’t impose constraint?
    

    import Ecto.Changeset
    
    alias Rumbl.Repo
    
    alias Rumbl.Multimedia.Video
    alias Rumbl.Multimedia.Category
    
    def create_category!(name) do
      %Category{}
      |> create_category_changeset(%{name: name})
      Repo.insert!(%Category{name: name}, on_conflict: :nothing)
    end
    
    def create_category_changeset(category, attrs) do
      category
      |> cast(attrs, [:name])
      |> validate_required([:name])
      |> unique_constraint(:name)
    end
    

    • Test it with
      

      alias Rumbl.Multimedia
      Multimedia.create_category!("Action")
      
      alias Rumbl.Multimedia.Category
      Multimedia.create_category_changeset(%Category{}, %{name: "Action"})
      

      • It should shows false, but the valid is true…
        
• How to delete the category table and create it again?
  
  • ref: How to delete/drop table?
    
  • Steps
    

    1. Generate a migration
       

       mix ecto.gen.migration drop_category
       

       • This command will contain a change function. By adding our custom logic into that change function, we could apply custom changes.
         

    2. Add change function to the migration
       

       defmodule Rumbl.Repo.Migrations.DropCategory do
         use Ecto.Migration
       
         def change do
           drop table("categories"), mode: :cascade
         end
       end
       

    3. Apply those changes
       

       mix ecto.migrate
       

• After we define our model using schema, how migration understand that model?
  
  • It doesn’t. We have to define the migration content by ourselves.
    
  • For example:
    

    • In our account/user.ex, we defined the following models
      

      defmodule Rumbl.Accounts.User do
        use Ecto.Schema
        import Ecto.Changeset
      
        schema "users" do
          field(:name, :string)
          field(:username, :string)
          timestamps()
        end
      end
      

    • mix ecto.gen.migration create_users
      
      • This will generate migration file named like: creating priv/repo/migrations/20180315023132_{createusers.exs}
        
    • In that create_users.exs, the change is empty. We have to define how to operation the database.
      So, we have to define how to create table, create attributes, and create indexes.
      
    • At last, we use mix ecto.migrate to migrate up the database.
      

• How the test cases could know our helper functions?
  
  • We define our helper functions in Rumbl.TestHelpers module.
    
  • We then import them globally in Rumbl.DataCase module in test/support/data_case.ex.
    

• How to drop a entire dev database and recreate all associated tables?
  
  1. mix ecto.drop rumbl_dev –force-drop
     
  2. mix ecto.create
     
  3. mix ecto.migrations (Use this command to check the current migrations available.)
     
  4. mix ecto.migrate
     

• How the js code in assets/js folder are available to the pages(template)
  
• See, chapter10
  

• How to quickly generate model (including schema and database change migration)
  

  mix phx.gen.schema Multimedia.Annotation annotations \
      body:text at:integer \
      user_id:references:users \
      video_id:references:videos
  

  • This command is create model annotation which including 4 fields
    
    • body
      
    • at
      
    • user_id
      
    • video_id
      
  • The result of running this command are two files one is migration_change. Another is annotation.ex file which changeset and schema.
    
  • Notice, the schema here doesn’t wire annotation with user or video.
    
  • To bind relationship between annotation with user and video. We need to do it with manually with careful decision.
    
  • Don’t forgot to modify the corresponding video or user’s schema to make space for annotation.